Image Processing Device Using Distance Information for Natural Superimposition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image processing technologies require high processing capacity to superimpose computer-generated images naturally onto real-space images, making it difficult to achieve natural appearances on devices with limited processing capacity, such as mobile terminals.

Innovation Solution

An image processing device and method that selects a removal region from an image based on distance information, specifies a complementary region from the image's background, and superimposes a complementary image generated from this region to enhance the natural appearance of the image by reducing processing complexity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If computer-generated images are superimposed on real-space images to achieve natural appearance, then image quality and naturalness are improved, but processing capacity requirements increase significantly

Engineering Contradiction:
Improveimage naturalnessVSAvoidprocessing capacity
Core Design Contradiction:
Manufacturing precisionVSPower

Solution Approach 1:

The patent segments the image processing task into distinct components: removing the object to be erased, generating a complementary image from the background region, and superimposing the complementary image. This segmentation allows each processing stage to be optimized independently, reducing the overall computational burden while maintaining natural appearance.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary actions by first removing the object to be erased and generating the complementary image before final superimposition. The complementary image is generated in advance from the background region, which simplifies the final composition step and reduces real-time processing requirements.

Inventive Principle:
Principle #10Preliminary action

2Manufacturing precision

If complex image processing is performed to achieve natural superimposition, then image quality is improved, but device complexity increases

Engineering Contradiction:
Improveimage naturalnessVSAvoidprocessing system complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The processing system is segmented into functional modules: an object removal unit, a complementary image generation unit, and a superimposition unit. Each module performs a specific function, making the overall complex task manageable through modular design and reducing system integration complexity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent creates a copy of the background region to generate the complementary image. This copying approach allows the system to work with replicated data rather than repeatedly accessing and processing the original complex scene, reducing computational complexity while maintaining image fidelity.

Inventive Principle:
Principle #26Copying

3Manufacturing precision

If object removal and complementary image generation are performed accurately, then image naturalness is improved, but calculation requirements increase

Engineering Contradiction:
Improveregion selection accuracyVSAvoidcalculation energy
Core Design Contradiction:
Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The patent extracts the background region surrounding the object to be erased and uses it to generate the complementary image. By taking out only the necessary background portions rather than processing the entire image, the calculation energy required is significantly reduced while maintaining accurate region selection.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies local quality processing by focusing computational resources on the specific regions that matter: the object boundary area and the immediate background surrounding it. Rather than uniformly processing the entire image, the system concentrates calculations on local areas that contribute most to natural appearance.

Inventive Principle:
Principle #3Local quality

Data Source

PatentUS11170511B2Image processing device, imaging device, and image processing method for replacing selected image area based on distance
Publication Date: 2021.11.09 SONY SEMICON SOLUTIONS CORP
  • US11170511B2 patent drawing
  • US11170511B2 patent drawing
  • US11170511B2 patent drawing

AI summary

The present disclosure relates to an image processing device, an imaging device, and an image processing method to superimpose a complementary image in such a manner that an appearance becomes more natural. From an image acquired by imaging of an arbitrary subject, a removal region selecting unit selects a region, in which an intended object to be removed is included, as a removal region with reference to distance information that is acquired when the image is imaged and that indicates a distance to the subject. Based on the distance information, a complementary region specifying unit specifies a complementary region to complement the removal region from a region including a background of the object to be removed in the image. An image superimposition processing unit superimposes a complementary image generated from the complementary region on a place from which the removal region is deleted in the image.