Virtual Light Source Effect Computation for Image Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional image processing techniques fail to effectively add a favorable illumination effect to images by considering the shooting environment and subject, potentially missing representation of gloss, luster, and enhanced contrast.

Innovation Solution

An image processing apparatus and method that estimates the ambient light conditions and determines the illumination direction and reflective characteristics of a virtual light source to compute and add a realistic illumination effect to images, incorporating features like diffuse and specular reflections based on the subject's properties and the shooting scene.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Illumination intensity

If a virtual light source is added to adjust brightness in shadow regions, then the luminance balance is improved, but the gloss and luster representation deteriorates

Engineering Contradiction:

Improvebrightness of shadow regionVSAvoidgloss and luster representation

Core Design Contradiction:

Illumination intensityVSEase of manufacture

Solution Approach 1:

The patent applies different lighting models to different regions of the subject. Specular reflection is applied to regions where gloss and luster should be enhanced, while diffuse reflection is applied to shadow regions for brightness adjustment. This local differentiation allows simultaneous improvement of brightness balance and gloss representation.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent dynamically adjusts parameters of the virtual light source including illumination direction, intensity distribution, and reflective characteristics based on the estimated ambient lighting conditions and subject properties. By changing these parameters adaptively, the system achieves both brightness correction and realistic gloss effect.

Inventive Principle:

Principle #35Parameter changes

2Stability of the object's composition

If a virtual light source is defined to illuminate the subject, then the stereoscopic effect is enhanced, but the natural appearance deteriorates

Engineering Contradiction:

Improvestereoscopic effectVSAvoidnatural appearance

Core Design Contradiction:

Stability of the object's compositionVSReliability

Solution Approach 1:

The system estimates the ambient light source characteristics from the captured image and uses this information to configure the virtual light source parameters. This feedback mechanism ensures that the added illumination complements rather than contradicts the existing lighting, maintaining natural appearance while enhancing stereoscopic effect.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent positions the virtual light source asymmetrically relative to the subject based on the estimated ambient lighting direction. This asymmetric positioning creates realistic shadow patterns and highlights that enhance depth perception while maintaining natural appearance, avoiding symmetric placement that would look artificial.

Inventive Principle:

Principle #4Asymmetry

3Productivity

If the illumination effect is computed without considering ambient light conditions, then the processing speed is improved, but the accuracy of illumination effect deteriorates

Engineering Contradiction:

Improveprocessing speedVSAvoidaccuracy of illumination effect

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system performs preliminary estimation of ambient light source characteristics (direction, intensity, color temperature) from the captured image before computing the virtual light source effect. This preliminary analysis enables accurate subsequent processing by providing essential context information, achieving both speed and accuracy through proper sequencing.

Inventive Principle:

Principle #10Preliminary action

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The solution enhances image quality by accurately simulating the illumination effect, reducing shadows and adding depth and luster, resulting in a more natural and aesthetically pleasing representation of the subject.

Implementation Method 1

a processing unit configured to compute the effect of the virtual light source based on the illumination direction of the virtual light source and the reflective characteristics of the subject determined by the determination unit

Methodology Applied

Scientific EffectDiffuse reflection: Reflection

Implementation Method 2

incorporating features like diffuse and specular reflections based on the subject's properties and the shooting scene

Methodology Applied

Scientific EffectSpecular reflection: Reflection

Data Source

PatentUS11503219B2Image processing apparatus, image capture apparatus, and control method for adding an effect of a virtual light source to a subject

Publication Date: 2022.11.15 CANON KK

AI summary

With respect to a subject included in an image, the illuminating condition by an ambient light source in an environment where the image was captured is estimated, and based on the estimation result, the effect of a virtual light source that was non-existent at the time of image capture is computed. More specifically, the effect of the virtual light source is computed using an illumination direction of the virtual light source and the reflective characteristics of the subject illuminated by the virtual light source, which have been determined based on the estimation result, and an image derived from addition of the effect of the virtual light source is output.