3D Photo Mesh Shadow Removal for Dynamic Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods fail to accurately remove shadows from three-dimensional photo meshes, which are essential for rendering scenes at different times of day or under varying conditions without requiring new aerial images, leading to inaccurate and unreliable results.
Innovation Solution
A method that detects shadows in 3D photo meshes using a processor, removes them to create a shadow-free texture, simulates real-time conditions, and renders images accordingly, allowing for dynamic content integration such as time of day, weather, and traffic without new aerial images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 2D image-based shadow detection techniques are used, then shadow detection can be performed, but accuracy and reliability are insufficient for 3D photo meshes
Solution Approach 1:
The patent transitions from 2D image-based shadow detection to 3D photo mesh-based shadow detection. By utilizing the third dimension (depth information) inherent in photo meshes, the system can accurately distinguish shadow regions from actual dark features in the scene, thereby resolving the accuracy and reliability issues of 2D methods.
Solution Approach 2:
The patent introduces a shadow mask as an intermediary element that stores shadow region information separately from the original photo mesh texture. This shadow mask acts as a mediator between the detection process and the final rendering, allowing accurate shadow identification without permanently altering the source imagery.
2Adaptability or versatility
If aerial images are captured at specific times, then photorealistic scenes can be created, but the scenes cannot be re-rendered at different times without new imagery
Solution Approach 1:
The patent extracts shadow information from the original aerial imagery and stores it separately in a shadow mask. This extraction allows the base photo mesh to be reused and re-rendered at different times of day by applying different lighting conditions, eliminating the need to capture new aerial images for each time period.
Solution Approach 2:
The patent performs shadow detection and creates the shadow mask in advance during the initial photo mesh creation process. This preliminary action prepares the data structure for future re-rendering operations, enabling flexible time-of-day simulations without requiring additional aerial image captures.
3Reliability
If shadows are retained in the photo mesh, then original imagery is preserved, but real-time condition simulation becomes inaccurate
Solution Approach 1:
The shadow mask serves as an intermediary that separates shadow information from the original photo mesh texture. The original imagery remains intact and preserved, while the shadow mask provides the necessary information for accurate real-time condition simulation by indicating which regions should be in shadow under different lighting conditions.
Solution Approach 2:
The patent segments the photo mesh data into two distinct components: the original texture (preserving original imagery) and the shadow mask (enabling accurate simulation). This segmentation allows both requirements to be satisfied simultaneously - the original image is preserved while the shadow information is available for realistic re-rendering.
Data Source
AI summary
Methods for rendering three-dimensional photo meshes having dynamic content include: (a) detecting a shadow in a three-dimensional photo mesh; (b) removing the shadow from the three-dimensional photo mesh to form a modified photo mesh having a shadow-free texture; (c) simulating a real-time condition in the modified photo mesh; and (d) rendering an image that shows an effect of the real-time condition. Systems for rendering three-dimensional photo meshes having dynamic content are described.