Image Compositing Using Luminance Masks to Reduce Characteristic Color Effects

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional image composition techniques, such as poisson image editing, face challenges in generating high-quality composite images due to the effects of characteristic colors like bangs and shadows, especially in unstable lighting environments, and struggle with accurate face detection, particularly for individuals wearing glasses or with rotated faces.

Innovation Solution

An information processing apparatus employing a detector to identify specific image regions using machine learning, generating mask images with higher luminance areas to reduce the impact of characteristic colors, and using these mask images for poisson image editing to composite images smoothly.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If poisson image editing is used for compositing, then smooth contour compositing is achieved, but characteristic colors like bangs and shadows still affect the composite quality

Engineering Contradiction:
Improvecomposite image qualityVSAvoidcharacteristic color effect
Core Design Contradiction:
Manufacturing precisionVSObject-affected harmful factors

Solution Approach 1:

The patent divides the image into multiple regions using generated masks, separating the target region (e.g., face) from surrounding regions containing characteristic colors like bangs and shadows. This segmentation allows selective processing where the target region is extracted and composited while the harmful characteristic color regions are excluded or treated differently, thus resolving the contradiction between smooth compositing and characteristic color interference.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies different processing qualities to different regions of the image. The target region receives high-quality processing with preserved detail and color, while surrounding regions containing characteristic colors are masked out or processed with different parameters. This local differentiation ensures that the composite image quality is high where needed while eliminating the harmful effects of characteristic colors from adjacent regions.

Inventive Principle:
Principle #3Local quality

2Measurement precision

If conventional face detection is used, then processing speed is maintained, but detection accuracy decreases for faces with glasses or rotation

Engineering Contradiction:
Improveface detection accuracyVSAvoidhandling of varied face conditions
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent performs preliminary processing steps including image normalization, rotation correction, and glass detection before the main face detection process. By preparing the image in advance (straightening rotated faces, removing glass artifacts, standardizing lighting), the subsequent face detection can achieve high accuracy across varied face conditions without requiring complex real-time adaptation during detection.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces intermediary processing steps and auxiliary algorithms between the input image and the main detection process. These intermediaries include preprocessing filters, normalization layers, and auxiliary detection models that handle specific challenging cases (glasses, rotation) before the primary face detection algorithm operates, thereby improving overall accuracy and adaptability.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Manufacturing precision

If multiple processing steps are added for mask generation and luminance adjustment, then composite image quality improves, but processing complexity increases

Engineering Contradiction:
Improveregion specification accuracyVSAvoidprocessing pipeline complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent combines multiple processing functions into integrated modules. The mask generation process is merged with luminance adjustment and region specification in a unified processing pipeline, where a single set of processing steps simultaneously achieves multiple objectives (creating masks, adjusting luminance, defining regions) rather than requiring separate independent processing stages, thus reducing overall complexity while maintaining precision.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent designs processing modules that perform multiple functions simultaneously. The mask generation algorithm also serves as a region specification tool and luminance adjustment reference, making it a multi-functional component that reduces the need for separate dedicated processing steps for each function, thereby improving region specification accuracy without proportionally increasing processing pipeline complexity.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS10911695B2Information processing apparatus, information processing method, and computer program product
Publication Date: 2021.02.02 KK TOSHIBA
  • US10911695B2 patent drawing
  • US10911695B2 patent drawing
  • US10911695B2 patent drawing

AI summary

An information processing apparatus includes a detector, a generator, and a compositor. The detector detects a first image region including a specific part of a subject from a first image obtained by photographing the subject. A generator generates a plurality of mask images for specifying a region to be composited with a second image including the first image region. The compositor composites the first image with the second image using the mask images. The mask images each include a region having higher luminance than luminance of the other regions out of regions of the first image, in a region in contact with another mask image or a region overlapping another mask image.