Photometric Stereo Surface Normal Correction via Light Quantity Feedback

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The photometric stereo method for calculating surface normal information is inaccurate when the irradiation light quantity fluctuates, especially due to differences in light source positions and environmental factors, leading to deviations from the assumed reflection model and incomplete surface normal calculation.

Innovation Solution

A processing apparatus and method that corrects the light source condition by detecting the brightness of multiple light sources with different positions, ensuring consistent light quantity across all sources, thereby maintaining accurate surface normal calculation using a corrector and detector system integrated with an image pickup apparatus.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the photometric stereo method is used to calculate surface normal information from multiple light sources, then surface normal information can be obtained, but calculation accuracy deteriorates when irradiation light quantity fluctuates among light sources

Engineering Contradiction:
Improvesurface normal calculation accuracyVSAvoidcalculation accuracy stability under light quantity fluctuation
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent introduces a feedback mechanism where the detected light quantity from each light source is used to adjust the luminance information. The detector measures the actual light quantity, and this measurement feeds back to the calculator to correct the surface normal computation, ensuring accurate results even when light quantities fluctuate

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent changes the parameter consideration by explicitly incorporating light quantity parameters into the surface normal calculation process. Instead of assuming equal light quantities, the system adjusts for actual light quantity variations by using detected brightness values as correction parameters in the photometric stereo equations

Inventive Principle:
Principle #35Parameter changes

2Loss of information

If multiple light sources with different positions are used to obtain surface normal information, then more comprehensive shape information can be obtained, but differences in irradiation light quantity among light sources cause large deviations in calculated surface normal

Engineering Contradiction:
Improvecompleteness of surface normal informationVSAvoidsurface normal calculation accuracy
Core Design Contradiction:
Loss of informationVSMeasurement precision

Solution Approach 1:

The system uses feedback from light quantity detection to correct the luminance information from each light source. The detector measures actual light quantities, and this information feeds back to adjust the photometric stereo calculation, ensuring that differences in light quantity among multiple light sources do not cause large deviations

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent modifies the calculation parameters by incorporating detected light quantity values into the surface normal computation. This allows the system to process luminance information from multiple light sources with different positions and intensities while maintaining accuracy through parameter adjustment

Inventive Principle:
Principle #35Parameter changes

3Measurement precision

If a plurality of images with different luminance information are captured for photometric stereo, then surface normal can be calculated, but the method assumes equal light quantity from all light sources which is often not met in practice

Engineering Contradiction:
Improvesurface normal calculation capabilityVSAvoidrobustness to light quantity variation
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent implements feedback by detecting actual light quantities from multiple light sources and using this information to correct the luminance data before surface normal calculation. This feedback loop allows the system to maintain the photometric stereo methodology while adapting to real-world variations in light quantity

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system changes the approach by treating light quantity as a variable parameter rather than a fixed assumption. By detecting and incorporating light quantity parameters into the calculation, the method becomes adaptable to different lighting conditions while maintaining surface normal calculation capability

Inventive Principle:
Principle #35Parameter changes

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach ensures precise calculation of surface normal information even when light quantity fluctuations occur, improving the accuracy of surface normal determination in image processing systems.

Implementation Method 1

detects brightness of each of the light sources

Methodology Applied
Scientific EffectBrightness detection:

Implementation Method 2

corrects luminance information of images on the basis of a detection result of the detector

Methodology Applied
Scientific EffectLight quantity correction:

Implementation Method 3

reflected light of the object has each component of specular reflected light and diffusion reflected light

Methodology Applied
Scientific EffectLight reflection: Reflection

Implementation Method 4

A Lambert reflection model in dependence upon a Lambert's cosine law is often used as the reflectance characteristics of the object

Methodology Applied
Scientific EffectLambert's cosine law:

Implementation Method 5

The specular reflected light is regular reflection on an object surface, and indicates Fresnel reflection in dependence upon a Fresnel equation on the object surface

Methodology Applied
Scientific EffectFresnel reflection:

Data Source

PatentUS10346959B2Processing apparatus, processing system, image pickup apparatus, processing method, and non-transitory computer-readable storage medium
Publication Date: 2019.07.09 CANON KK
  • US10346959B2 patent drawing
  • US10346959B2 patent drawing
  • US10346959B2 patent drawing

AI summary

A processing apparatus includes a corrector that, when the object is sequentially irradiated with light from three or more light sources with mutually different positions, corrects a light source condition to calculate surface normal information of an object on the basis of brightness of each of the light sources.