Image Processing Apparatus Depth Map Sharpness Control

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image processing methods fail to accurately convey a three-dimensional appearance in two-dimensional images due to inadequate control over sharpness and image quality degradation, leading to discomfort or enhanced image quality issues.

Innovation Solution

An image processing apparatus and method that obtain distance information from an in-focus plane for each pixel, adjust image processing conditions based on output characteristics, and selectively apply image processing to specific spatial frequency bands to maintain or restore the intended three-dimensional appearance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If depth information is estimated and calculated from an image itself, then processing can be performed without additional hardware, but the depth information cannot be correctly obtained depending on the characteristic or composition of the image

Engineering Contradiction:
Improveprocessing method simplicityVSAvoiddepth information accuracy
Core Design Contradiction:
Ease of manufactureVSMeasurement precision

Solution Approach 1:

The patent introduces a depth map as an intermediary representation that stores depth information for each pixel. This depth map is then used to control the sharpness processing parameters, allowing accurate depth-based sharpness adjustment without requiring complex real-time calculations during image processing. The depth map serves as a pre-computed mediator between the original image and the final processed output.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If image processing is applied to enhance three-dimensional appearance, then the sharpness and depth perception are improved, but image quality degradation factors such as noise are enhanced

Engineering Contradiction:
Improvethree-dimensional appearance qualityVSAvoidimage quality degradation
Core Design Contradiction:
ReliabilityVSObject-generated harmful factors

Solution Approach 1:

The patent applies different sharpness processing parameters to different regions of the image based on their depth information. By using the depth map to determine local processing characteristics, the system enhances the three-dimensional appearance in the in-focus region while preserving or suppressing noise in the out-of-focus region. This local differentiation allows selective quality enhancement without amplifying degradation factors across the entire image.

Inventive Principle:
Principle #3Local quality

3Productivity

If sharpness control is performed without considering human visual characteristics, then processing can be applied uniformly, but a sense of discomfort occurs in the three-dimensional appearance

Engineering Contradiction:
Improveprocessing efficiencyVSAvoidvisual comfort
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The patent dynamically changes the sharpness processing parameters based on the depth information and human visual characteristics. By adjusting parameters such as the degree of sharpness enhancement and noise suppression according to the depth map and visual perception models, the system achieves both efficient processing and visually comfortable output. This parameter adaptation ensures that the processed image conforms to human visual expectations.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS11645774B2Image processing apparatus, image processing method, and non-transitory computer-readable storage medium
Publication Date: 2023.05.09 CANON KK
  • US11645774B2 patent drawing
  • US11645774B2 patent drawing
  • US11645774B2 patent drawing

AI summary

An image processing apparatus comprises: an obtaining unit configured to obtain an image and distance information concerning a distance from an in-focus plane, which corresponds to each pixel included in the image; a setting unit configured to set an image processing condition according to the distance information based on an output characteristic of an output apparatus concerning a sharpness; and a processing unit configured to perform image processing for the image using the distance information obtained by the obtaining unit and the image processing condition set by the setting unit, wherein the processing unit changes, in accordance with the distance information, a band of a spatial frequency of the image to which the image processing is applied.