Image Processing Apparatus Viewpoint Position Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems struggle to accurately represent reflections on real objects when capturing images of both a CG image displayed on a display device and the real object, particularly in virtual production setups using video walls.
Innovation Solution
An image processing apparatus and method that generates images using a three-dimensional model, where the viewpoint position within the image capture range differs from the viewpoint position outside the capture range, allowing for accurate representation of reflections by combining camera and real object perspective images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a single viewpoint position is used for generating CG images, then the image generation process is simple, but reflections on real objects cannot be accurately represented
Solution Approach 1:
The image display area is divided into two segments: an image capture range area and an area outside the capture range. Different viewpoint positions are applied to each segment - the camera viewpoint position for the capture range area and the real object viewpoint position for the outside area. This segmentation allows accurate reflection representation while managing complexity through structured division.
Solution Approach 2:
Different viewpoint positions are applied locally to different areas of the image. The area within the image capture range uses the camera viewpoint position to maintain consistency with captured reflections, while the area outside the capture range uses the real object viewpoint position. This local differentiation enables accurate reflection representation without requiring a completely complex system.
2Manufacturing precision
If CG images are generated with consistent viewpoint position, then image generation is efficient, but reflections on real objects appear inaccurate
Solution Approach 1:
The image is segmented into capture range and non-capture range areas, allowing efficient generation through standardized processes for each segment while achieving accurate reflections through the differentiated viewpoint positions. This segmentation maintains efficiency by avoiding complete re-generation while improving reflection accuracy.
Solution Approach 2:
The viewpoint position parameter is changed based on the spatial location within the image. By adjusting this key parameter between the camera viewpoint position and real object viewpoint position for different areas, the system achieves accurate reflection representation without requiring a complete redesign of the generation process, thus maintaining efficiency.
3Manufacturing precision
If the same viewpoint position is used for all image areas, then the processing flow is simple, but light direction alignment for reflections is incorrect
Solution Approach 1:
The image display area is segmented into capture range and outside areas, with appropriate viewpoint positions assigned to each. This segmentation enables correct light direction alignment for reflections by using the camera viewpoint position for captured areas and the real object viewpoint position for other areas, while managing complexity through clear spatial division.
Solution Approach 2:
Different viewpoint positions are applied locally to different spatial regions. The camera viewpoint position is used locally for the capture range area to ensure correct light direction alignment with actual captured reflections, while the real object viewpoint position is applied locally to the outside area. This local quality differentiation achieves accurate light alignment without requiring system-wide complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present technology relates to an image processing apparatus, an image processing method, and a program that are able to accurately represent reflections appearing on a real object in an image obtained by capturing an image displayed on a display device and an image of the real object. A selection section generates a CG image that is to be displayed on a video wall and captured by a video camera by use of a three-dimensional model. A viewpoint position of an area of the CG image within an image capture range of the video camera is different from a viewpoint position of an area outside the image capture range. The present technology can be applied, for example, to a PC of a virtual production imaging system that includes a video camera, a video wall on which a CG image captured by the video camera is displayed, a PC, a video wall controller, and a depth sensor.