Relightable Texture for Accurate Shadow Detail Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing free-viewpoint rendering systems struggle with accurately relighting scenes under arbitrary lighting conditions, particularly failing to correctly render small shadow regions due to insufficient consideration of ambient occlusion.
Innovation Solution
The method involves separating an initial texture into a lighting estimate and a colour estimate, filtering the lighting estimate to generate a shadow-detail estimate, and using these components to determine a relightable texture that retains shadow regions indicative of ambient occlusion, ensuring accurate shadowing even under different lighting conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the initial texture is separated into lighting estimate and colour estimate for independent processing, then the relighting flexibility is improved, but the processing complexity increases
Solution Approach 1:
The patent segments the initial texture into two distinct components: a lighting estimate and a colour estimate. This segmentation allows independent processing of each component, enabling flexible relighting while maintaining colour integrity. The lighting estimate captures illumination information that can be modified for different viewing conditions, while the colour estimate preserves the inherent colour properties of the scene.
2Measurement precision
If the lighting estimate is filtered with a high-pass filter to retain shadow regions, then the shadow detail accuracy is improved, but the computational load increases
Solution Approach 1:
The patent applies a high-pass filter to the lighting estimate to selectively retain high-frequency shadow detail information while discarding low-frequency components. This partial action approach focuses computational resources only on preserving the critical shadow regions that contribute to realism, rather than processing the entire lighting information equally. The filtering operation efficiently isolates the shadow-detail estimate needed for accurate relighting.
3Adaptability or versatility
If textures are extracted from multiple camera views, then the novel viewpoint rendering capability is improved, but the lighting artefacts in the texture increase
Solution Approach 1:
The patent extracts and isolates the lighting information from the multi-view texture synthesis process by creating a separate lighting estimate component. This extraction separates the harmful lighting artefacts that vary between camera views from the underlying colour and geometric information. The lighting estimate can then be processed independently to remove or adjust these artefacts while preserving the beneficial multi-view rendering capability.
Data Source
AI summary
A model of a scene of an image (e.g. a frame of a video sequence) is generated from one or more views of the scene captured from one or more different camera viewpoints. An initial texture for applying to the model is derived from the one or more views of the scene. The initial texture is separated into a lighting estimate and a color estimate, which may be orthogonal and which may be processed independently. The lighting estimate is filtered with a high-pass filter to thereby determine shadow regions of the scene which are regions of detailed shadow which are likely to be caused by ambient occlusion in the scene and which are therefore retained when the texture is relit for rendering the image. A shadow-detail estimate (or “dark map”) is provided which indicates one or more shadow regions of the texture which are to remain in shadow when the image is rendered.