3D HDR Radiance Fields for Novel View Rendering Under Varying Exposure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional HDR imaging technologies struggle to generate 3D images for intermediate viewpoints and fail to consider varying photometric conditions, leading to noise and performance deterioration.
Innovation Solution
A method for generating 3D HDR radiance fields from LDR images captured at various viewpoints, incorporating photometric conditions, involves constructing 3D HDR radiance fields, ray marching, tone mapping, and parameter adjustment to minimize loss, enabling image generation for novel views and varying photometric conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional HDR imaging technology is used to restore HDR images, then HDR images for given viewpoints can be restored, but images for intermediate viewpoints cannot be restored
Solution Approach 1:
The patent transitions from 2D HDR image restoration to 3D radiance field representation. By encoding radiance information in three-dimensional space with directional components, the system enables novel view synthesis while maintaining HDR quality for both given and intermediate viewpoints.
Solution Approach 2:
The patent creates a 3D radiance field model that copies and reconstructs scene radiance information from input images. This learned radiance field can then generate HDR images for viewpoints not present in the original input, effectively copying the radiance properties to novel views.
2Ease of manufacture
If conventional 3D volume rendering technology is used, then 3D images can be generated, but photometric conditions such as exposure and white balance are not considered
Solution Approach 1:
The patent incorporates photometric parameters including exposure, white balance, and camera response function into the radiance field model. By learning and storing these parameter variations across different viewpoints, the system accurately reproduces photometric conditions when generating HDR images for novel views.
3Device complexity
If conventional volume rendering assumes small viewpoint-dependent color changes, then training is simplified, but rapid color changes cause incorrect 3D geometric information and noise
Solution Approach 1:
By moving to a full 3D radiance field representation with directional radiance components, the patent can accurately capture rapid viewpoint-dependent color changes without making simplifying assumptions. The additional dimensional information allows the model to distinguish between geometric structure and color variation.
Data Source
AI summary
An image processing device may perform a method for generating 3D HDR radiance fields, and the method may include: receiving an input of low dynamic range (LDR) images, which are captured at various viewpoints; constructing the 3D high dynamic range (HDR) radiance fields from the LDR images captured at the various viewpoints; generating an HDR image by ray marching on a basis of the 3D HDR radiance fields; generating a tone-mapped LDR image from the HDR image through a tone mapping module; calculating a loss value on a basis of the LDR images captured at the various viewpoints and the tone-mapped LDR image; and modifying parameters of the 3D HDR radiance fields and parameters of the tone-mapping module, so as to maximally minimize the loss value. The LDR images captured at the various viewpoints may be images captured under photometric conditions different from each other.