Virtual Light Source Rendering with Normal Line Error Smoothing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The photometric stereo method fails to accurately acquire normal line information due to specularly reflected light, shadow parts, and interreflection, leading to image quality degradation in virtual light source images.
Innovation Solution
An image processing apparatus that acquires light source information, normal line information, and normal line error information, and generates corrected gain information by smoothing processing in error regions to improve image quality by applying appropriate gains to virtual light source effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the photometric stereo method is used to acquire normal line information, then normal line information can be obtained from captured images, but image quality degrades in regions with specular reflection, shadow, or interreflection due to errors from the Lambertian reflectance model
Solution Approach 1:
The image processing apparatus segments the image into reliable regions and error regions based on normal line information quality. By identifying regions where the Lambertian reflectance model fails (specular reflection, shadow, interreflection), the system applies different processing strategies to each segment, preserving accuracy in reliable regions while mitigating errors in problematic regions through selective handling.
Solution Approach 2:
The system applies local quality control by generating virtual light source images with spatially varying reliability characteristics. In regions with normal line errors, the system adjusts the rendering process to account for reduced reliability, while maintaining full quality in regions where normal line information is accurate. This creates an output image with non-uniform quality that matches the underlying data reliability.
2Area of stationary object
If normal line information including error regions is used to generate virtual light source images, then complete image coverage is achieved, but shadow appearance becomes unnatural in error regions
Solution Approach 1:
The system performs preliminary identification of error regions before generating the final virtual light source image. By pre-processing the normal line information to flag regions with specular reflection, shadow, or interreflection errors, the system can prepare appropriate correction strategies in advance, ensuring that when the virtual light source image is generated, error regions are handled specially to maintain natural shadow appearance.
Solution Approach 2:
The system introduces an intermediary reliability assessment layer between the normal line information and the virtual light source image generation. This intermediary component evaluates the quality of normal line information in each region and mediates the rendering process accordingly, adjusting shadow calculations in error regions to produce natural-looking results despite the underlying data problems.
Data Source
AI summary
An acquisition unit acquires light source information specifying a virtual light source, normal line information indicating a normal line distribution in a shooting range, and normal line error information indicating a normal line error region in which an error has occurred in the normal line distribution. A first generation unit generates first gain information indicating a first gain amount distribution, based on the light source information and the normal line information. A second generation unit generates second gain information indicating a second gain amount distribution, by performing smoothing processing on a gain amount of a region, which corresponds to the normal line error region, in the first gain amount distribution. An addition unit adds an effect of the virtual light source to an image of the shooting range based on the second gain amount distribution.