Virtual Light Source Color Control for Shadow Regions

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image processing methods that add virtual lighting to correct shadows often result in unnatural coloration, especially when sunlight hits objects obliquely, causing shadow regions to appear bluish due to diffused light, which is not corrected by brightness adjustments alone.

Innovation Solution

An image processing apparatus and method that identifies target colors and regions for virtual light sources, adjusts the color of the virtual light source to match the target color, and controls the white balance of the target region to ensure the color of the virtual light source approaches the target color, thereby suppressing unnatural coloring.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Illumination intensity

If virtual lighting processing that corrects only brightness is carried out on shadow regions, then the brightness of shadow regions is improved, but the color of shadow regions becomes unnatural

Engineering Contradiction:
Improvebrightness of shadow regionsVSAvoidcolor accuracy of shadow regions
Core Design Contradiction:
Illumination intensityVSManufacturing precision

Solution Approach 1:

The patent changes the parameters of the virtual light source (color temperature, color tone) to match the target color identified from directly lit regions. By adjusting these parameters, the processing unit adds virtual light that corrects both brightness and color simultaneously, resolving the contradiction between improving shadow brightness and maintaining color accuracy

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The system identifies the target color from regions directly lit by the light source and uses this information as feedback to adjust the virtual light source parameters. This feedback mechanism ensures that the corrected shadow regions match the color characteristics of directly lit regions, preventing unnatural coloration while improving brightness

Inventive Principle:
Principle #23Feedback

2Manufacturing precision

If the color of the virtual light source is adjusted to match the target color, then the natural appearance of shadow regions is improved, but the complexity of the image processing increases

Engineering Contradiction:
Improvecolor accuracy of shadow regionsVSAvoidcomplexity of image processing
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent segments the image into different regions (shadow regions and directly lit regions) and applies different processing to each. By identifying target colors from directly lit regions and applying virtual lighting only to shadow regions with matched color parameters, the system achieves accurate color correction without unnecessarily processing the entire image, thus managing complexity effectively

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system copies the color characteristics from regions directly lit by the light source and applies them to shadow regions through the virtual light source. This copying approach simplifies the processing by reusing existing color information from the image rather than requiring complex color analysis or external color data

Inventive Principle:
Principle #26Copying

Data Source

PatentUS11115637B2Image processing apparatus, image capturing apparatus, image processing method, and storage medium
Publication Date: 2021.09.07 CANON KK
  • US11115637B2 patent drawing
  • US11115637B2 patent drawing
  • US11115637B2 patent drawing

AI summary

There is provided an image processing apparatus comprising. An identification unit identifies a target color that a pixel of a shot image corresponding to an object having a specific color is to have as a result of white balance processing performed on the shot image. A determination unit determines a target region of a virtual light source in the shot image. An image processing unit performs image processing for adding an effect of virtual light irradiation by the virtual light source to the target region. A control unit controls a color of the virtual light source such that a color close to the target color approaches the target color in the target region.