Distance Layer Mapping for Intuitive CG Image Compositing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing devices struggle to intuitively present distance information for video synthesis, making it difficult for photographers to understand where computer graphics can be easily inserted, and they lack the ability to classify images based on distance information effectively, especially when few subjects are present.
Innovation Solution
An image processing device that utilizes a sensor with an image surface phase difference system to generate distance information, which is then converted into a distance layer map using a histogram or image recognition, allowing for intuitive understanding and effective classification of layers for inserting computer graphics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If distance information is output as numerical values or heat maps, then distance measurement is achieved, but it is difficult for photographers to intuitively understand where CG can be easily inserted
Solution Approach 1:
The patent applies color-coded distance layer maps where different colors represent different distance ranges. This visual encoding transforms abstract distance numerical values into intuitive color-based spatial information, allowing photographers to quickly understand where CG elements should be inserted based on color-coded depth zones without needing to interpret numerical distance data or grayscale heat maps.
Solution Approach 2:
The patent transitions from one-dimensional numerical distance values or grayscale heat maps to a two-dimensional color-coded spatial map that overlays distance information directly on the image plane. This dimensional transformation provides both spatial context and distance information simultaneously, making it intuitive to understand where CG elements should be placed in three-dimensional space.
2Extent of automation
If images are classified using only distance information, then distance-based classification is achieved, but images with few subjects are not effectively classified and CG insertion opportunities are missed
Solution Approach 1:
The patent creates a classification system that serves multiple functions simultaneously: it classifies images based on distance layer distribution, identifies subjects within those layers, and determines CG insertion suitability. This multi-functional approach ensures that even images with few subjects are properly classified by considering both spatial distance distribution and subject presence, rather than relying solely on distance metrics.
Solution Approach 2:
The patent implements a feedback mechanism where the classification results are used to identify images suitable for CG insertion, and this information feeds back into refining the classification criteria. The system learns from classification outcomes and adjusts its criteria to better identify images with few subjects that still represent good CG insertion opportunities, improving overall classification reliability.
3Ease of manufacture
If chroma key synthesis is used for video synthesis, then background replacement is achieved, but it requires preparing specific color backgrounds and performing complex smoothing and illumination adjustments
Solution Approach 1:
The patent replaces the mechanical chroma key method (requiring physical green or blue backgrounds and manual smoothing operations) with an automated optical system using imaging elements that directly capture distance information. This substitution eliminates the need for specific color backgrounds and manual illumination adjustments, as the system automatically separates subjects from backgrounds based on captured distance data.
Solution Approach 2:
The patent enables the imaging system to automatically perform background separation and distance-based layer classification without requiring manual chroma key setup or post-processing adjustments. The system self-services by capturing both image and distance information simultaneously and automatically generating distance layer maps, eliminating the need for photographers to perform complex chroma key preparation work.
4Measurement precision
If separate distance sensor devices are used for measuring distance, then distance measurement is achieved, but calibration work is required before imaging due to differences in angle of view and resolution
Solution Approach 1:
The patent merges the distance measurement function directly into the imaging element itself, combining what were previously separate functions (imaging and distance sensing) into a single integrated system. This integration eliminates the need for separate calibration procedures between the imaging device and distance sensor, as both functions share the same optical path and sensor plane, automatically ensuring consistent angle of view and resolution characteristics.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Facilitates intuitive setting of layer information based on distance, enabling easier insertion of computer graphics by providing a clear understanding of suitable layers and improving the classification of images based on distance information.
Implementation Method 1
an imaging element having a distance measurement function of an image surface phase difference system
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
An image processing device that can facilitate setting related to layer information based on distance information is provided. The image processing device includes an image acquisition unit configured to acquire an image including a subject through a lens unit, a distance information acquisition unit configured to acquire distance information indicating a distance to the subject, a layer information generation unit configured to generate layer information on a layer for each distance based on the distance information, and a setting unit configured to set a reference for generating the layer information and switch display of a setting value capable of being set in accordance with the lens information of the lens unit.