Free-Viewpoint Image Generation with Azimuth-Based Lighting Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing three-dimensional modeling technologies, such as NeRF, require fixed lighting environments for shooting teacher images, making it difficult to change lighting conditions and luster in generating free-viewpoint two-dimensional images, often necessitating professional expertise.
Innovation Solution
An image generation apparatus and method that includes a pre-trained optical training/inference part and a reflection azimuth calculator, allowing control of lighting environment and luster by varying an angle control parameter to generate free-viewpoint two-dimensional images from a fixed lighting environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed lighting environment is used for shooting teacher images in NeRF, then the three-dimensional modeling can be trained effectively, but the lighting environment and luster cannot be controlled when generating free-viewpoint two-dimensional images
Solution Approach 1:
The patent pre-trains the optical training/inference part with azimuth information during the initial NeRF training phase. This preliminary action stores optical properties and lighting characteristics in the spatial structure, enabling later manipulation of lighting environment and luster without requiring additional shooting setups.
Solution Approach 2:
The patent introduces an angle control parameter that modifies the observation azimuth when generating free-viewpoint images. By changing this parameter, the system can control lighting environment and luster effects dynamically, transforming the fixed lighting constraint into a controllable variable.
2Manufacturing precision
If professional photographers are required to set up shooting environments, then image quality can be ensured, but the process becomes complex and time-consuming
Solution Approach 1:
The system performs self-service by automatically extracting and storing optical information during the initial training phase. The pre-trained optical training/inference part autonomously processes azimuth information and spatial structure, eliminating the need for manual professional setup while maintaining image quality through algorithmic optimization.
Solution Approach 2:
The patent replaces the mechanical process of manual lighting setup and camera positioning with an computational system. The reflection azimuth calculator and optical training/inference part use algorithms to simulate and control lighting effects, substituting physical shooting environment manipulation with digital processing.
3Adaptability or versatility
If reshooting is performed to change lighting conditions, then lighting environment can be adjusted, but time and resources are wasted
Solution Approach 1:
The patent performs preliminary training with comprehensive azimuth information coverage during the initial shooting phase. This pre-training stores multiple lighting perspective data points in the spatial structure, enabling later retrieval and manipulation of different lighting conditions without requiring additional shooting sessions.
Solution Approach 2:
The patent transforms the static fixed lighting environment into a dynamic controllable system. The angle control parameter allows real-time adjustment of observation azimuth and lighting effects during image generation, enabling flexible lighting condition changes without physical reshooting.
Data Source
AI summary
An image generation apparatus generates a free-viewpoint two-dimensional image of a three-dimensional scene, a spatial structure and information on optics of which have been trained and comprises a pre-trained optical training/inference part and a reflection azimuth calculator, wherein the optical training/inference part receives azimuth information and extracts optical information by utilizing at least the azimuth information, and the reflection azimuth calculator receives an angle control parameter as an input, performs a calculation of an observation azimuth and the angle control parameter to at least a partial region within a reconstructed space based on the trained spatial structure to generate the azimuth information, and outputs the azimuth information to the optical training/inference part.