Shadow-Free 3D Map Generation for Autonomous Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing map generation techniques for autonomous vehicles often include shadows, which can lead to inaccurate maps, confusion, and difficulties in localization and segmentation due to varying lighting conditions and perspectives.
Innovation Solution
The method involves identifying shadow regions in images using machine learning algorithms and 3D mesh geometry, then removing or minimizing shadows by blending or duplicating image data from other regions with similar semantic classifications and lighting conditions, creating a shadow-free map that can simulate arbitrary lighting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If shadow regions are included in map data, then the map reflects actual captured image data, but the map becomes distracting and causes confusion due to varying lighting conditions
Solution Approach 1:
The patent extracts and removes shadow regions from captured images before generating map data. The system identifies shadow regions using machine learning algorithms and 3D mesh geometry, then excludes these regions from the final map composition, thereby eliminating the harmful effect of shadows while preserving accurate representation of actual environment features.
Solution Approach 2:
The patent applies different processing quality to different regions of the image. Non-shadow regions are processed with standard mapping quality, while shadow regions are identified and handled differently (removed or replaced). This local differentiation ensures that only problematic shadow areas are affected, maintaining overall map reliability while eliminating distraction.
2Reliability
If multiple images are processed to remove shadows, then shadow-free maps are generated, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary identification of shadow regions using machine learning algorithms trained to detect shadows, and uses pre-computed 3D mesh geometry to predict shadow areas. By identifying and marking shadow regions before the actual map composition process, the system avoids time-consuming shadow removal during final map generation, thereby reducing overall processing time while maintaining accuracy.
Solution Approach 2:
The patent uses 3D mesh representations as a copy or simplified model of the environment geometry to predict shadow regions. Instead of processing multiple full-resolution images to determine shadow areas, the system copies the essential geometric information into a 3D mesh model, which is then used to efficiently calculate and identify shadow regions without the computational burden of full image processing.
3Reliability
If shadow regions are removed by blending or duplicating image data, then accurate shadow-free maps are created, but memory requirements and processing complexity increase
Solution Approach 1:
The patent extracts shadow regions from the image data and removes them from the map composition process. Rather than attempting to repair or reconstruct shadow areas through complex blending operations, the system simply excludes identified shadow regions from the final map, significantly reducing processing complexity and memory requirements while maintaining map accuracy.
Solution Approach 2:
The patent uses temporary, computationally inexpensive representations of shadow regions (such as binary masks or simple geometric overlays) during the processing stage, then discards these temporary objects after shadow removal is complete. This approach avoids the need for complex, memory-intensive blending or duplicating operations, reducing both processing complexity and memory requirements.
Data Source
AI summary
Techniques for generating maps without shadows are discussed herein. A plurality of images can be captured by a vehicle traversing an environment representing various perspectives and/or lighting conditions in the environment. A shadow within an image can be identified by a machine learning algorithm trained to detect shadows in images and/or by projecting the image onto a three-dimensional (3D) map of the environment and identifying candidate shadow regions based on the geometry of the 3D map and the location of the light source. Shadows can be removed or minimized by utilizing blending or duplicating techniques. Color information and reflectance information can be added to the 3D map to generate a textured 3D map. A textured 3D map without shadows can be used to simulate the environment under different lighting conditions.