New Viewpoint Image Generation Using Albedo From Few-Shot 2D Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies like NeRF struggle to generate high-quality new viewpoint images using a small number of 2D images captured under varying lighting conditions, leading to degraded image quality.
Innovation Solution
An apparatus and method that extracts intrinsic color information (albedo) from 2D images, minimizing albedo consistency loss through geometric alignment and projective transformation, to generate reliable new viewpoint images using few-shot 2D images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If NeRF is used to generate new viewpoint images from multiple 2D images, then the quality of generated images is improved, but the number of required input images increases to more than 100
Solution Approach 1:
The patent extracts albedo (intrinsic color information) from input images to separate it from lighting conditions. By extracting only the essential color information without lighting variations, the system can generate high-quality new viewpoint images using fewer input images (few-shot), thus resolving the contradiction between image quality and number of required images
Solution Approach 2:
The patent changes the parameter representation by transforming images into albedo space, where lighting variations are removed. This parameter transformation allows the system to work with fewer images while maintaining quality, as albedo represents the fundamental color property independent of illumination conditions
2Adaptability or versatility
If NeRF is used to generate new viewpoint images from 2D images captured under different lighting conditions, then the adaptability to varying capturing conditions is improved, but the quality of generated images deteriorates
Solution Approach 1:
By extracting albedo from input images, the system separates intrinsic color information from lighting conditions. This extraction enables the system to adapt to different lighting conditions while maintaining consistent image quality, as the albedo representation is invariant to illumination variations
Solution Approach 2:
The patent transforms the image representation to albedo space, changing the parameter from raw pixel values (affected by lighting) to intrinsic color values (invariant to lighting). This parameter change allows the system to handle varying capturing conditions without quality degradation
3Quantity of substance
If RegNeRF is used to generate new viewpoint images with a small number of input images, then the number of required images is reduced, but the quality of generated images deteriorates when input images have different capturing conditions
Solution Approach 1:
The patent extracts albedo from few-shot input images to obtain lighting-invariant color information. This extraction compensates for the limited number of images by removing the variability introduced by different lighting conditions, thereby maintaining image quality despite using only a small number of inputs
Data Source
AI summary
The present disclosure relates to an apparatus and a method for generating an image, which may generate a new viewpoint image using few-shot 2D images and is characterized by extracting and learning albedo representing intrinsic color information from the few-show 2D input images, intrinsically decomposing a color value synthesized in the form of patch from a new viewpoint using the few-shot 2D input images and extracting patch-wise albedo during learning, perform geometric alignment necessary for pixel correspondence between a new viewpoint image and a selected 2D input image using a depth value synthesized in the form of patch from the new viewpoint, and generating the new viewpoint image by calculating an albedo consistency loss between the patch-wise albedo.