Processing Apparatus for Normal Error Detection in Rendered Images
Solution Overview
Problem
The photometric stereo method used for generating rendered images often results in errors when calculating the surface normal of objects with reflection characteristics different from the assumed Lambert diffuse reflection model, leading to incorrect and degraded rendered images, particularly for objects with few diffuse reflections like metals or transparent bodies.
Innovation Solution
A processing apparatus and method that acquires input images and normal information, detects areas with significant normal errors, and generates corrected rendered images by replacing erroneous areas with input image data, using multiple light sources and advanced rendering techniques to account for various reflection characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the photometric stereo method is used to calculate surface normal, then the rendering processing can be performed, but a normal error occurs for objects with reflection characteristics different from the assumed Lambert diffuse reflection model
Solution Approach 1:
The patent creates a rendered image copy of the input image using the acquired normal information and lighting conditions. This rendered copy is then compared with the original input image to detect areas where normal information may be erroneous, allowing identification and correction of inaccurate surface normal calculations without redoing the entire photometric stereo process
Solution Approach 2:
The patent implements a feedback mechanism where the rendered image is compared with the original input image to generate an error map. This error detection feedback allows identification of regions with large normal errors, enabling selective correction or refinement of normal information in those specific areas while maintaining the overall rendering workflow
2Device complexity
If the Lambert diffuse reflection model is assumed for photometric stereo, then the calculation is simplified, but particularly large normal errors occur for objects with few diffuse reflections such as metal or transparent bodies
Solution Approach 1:
The patent changes the approach from assuming a single reflection model to using multiple reflection models (diffuse, specular, and transparent) with different parameters. By selecting and applying appropriate reflection models based on the detected error magnitude and object characteristics, the system maintains calculation simplicity while improving reliability for different material types
Solution Approach 2:
The patent combines multiple reflection models (diffuse Lambertian, specular, and transparent reflection models) into a composite approach. Instead of relying solely on the Lambert model, the system integrates multiple physical models to accurately represent diverse object surfaces, thereby improving normal information reliability for metals, transparent bodies, and other non-Lambertian materials
3Productivity
If normal information is acquired using photometric stereo, then rendered images can be generated, but the rendered images become incorrect and degraded when normal errors occur
Solution Approach 1:
The patent extracts and isolates the erroneous regions by comparing the rendered image with the original input image. Areas with large discrepancies are identified and separated from the rest of the image, allowing targeted correction of only those specific regions with normal errors while preserving the correct portions of the rendered image
Solution Approach 2:
The patent substitutes the traditional approach of directly using photometric stereo results with a rendering-based verification system. Instead of relying solely on the mathematical photometric stereo calculation, the system uses physical rendering with the acquired normal information to generate a visual representation, then compares this rendered output with the actual captured image to detect and correct errors
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables the generation of high-quality rendered images by accurately determining surface normals and reflectance, even for objects with non-Lambertian reflection characteristics, thereby reducing errors and improving image quality.
Implementation Method 1
an image sensor configured to photoelectrically convert an optical image formed via an imaging optical system and to output image data corresponding to an input image
Data Source
AI summary
A processing apparatus includes an input image acquirer configured to acquire an input image obtained by capturing an object, a normal information acquirer configured to acquire normal information of the object, a first renderer configured to generate a first rendered image about the object based on a light source condition in capturing the input image and the normal information, and an area detector configured to detect a target area corresponding to a portion where an error of the normal information in the object is larger than a predetermined value based on the input image and the first rendered image.