In-Camera VFX Background Rendering with Real Light Source Influence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional in-camera VFX video processing fails to naturally incorporate the influence of real-world light sources into virtual backgrounds, resulting in unnatural visual effects.
Innovation Solution
An image processing apparatus and method that acquires information about real-space light sources and generates background images based on a three-dimensional virtual space model, incorporating the effects of these light sources to create a more realistic visual experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional in-camera VFX processing is used to capture subjects with camera-position-dependent background images, then the background image generation is simple and fast, but the visual realism deteriorates because light source influences are not incorporated
Solution Approach 1:
The patent merges the real-space light source information acquisition process with the virtual-space background image generation process. The light source information acquisition unit captures light source data from the real captured scene, and this information is directly integrated into the background image generation unit that renders virtual backgrounds, allowing light influences to naturally affect both real and virtual elements in the composite image
Solution Approach 2:
The patent introduces light source information as an intermediary element that mediates between the real captured scene and the virtual background. This intermediary carries light source characteristics (position, intensity, color) from the real scene and transfers them to influence the rendering of virtual objects, creating coherent lighting across the entire image
2Reliability
If light source information from real space is acquired and incorporated into virtual background generation, then visual realism is improved, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary acquisition of light source information from the real captured scene before generating the virtual background. By detecting and storing light source characteristics (position, intensity, color temperature) in advance, the system prepares the lighting data needed for realistic rendering, reducing the computational burden during the actual background generation phase
Solution Approach 2:
The patent applies light source influences selectively to different regions of the virtual background based on local lighting conditions. Rather than uniformly processing the entire virtual scene, the system applies light effects only to areas where real light sources have measurable influence, optimizing computational resources while maintaining visual realism
Data Source
AI summary
An image processing apparatus capable of causing the influence of a light source that is in a captured scene to affect a background image of in-camera VFX video, is disclosed. The apparatus acquires information relating to a light source existing in real space captured by an image capture apparatus and generates, based on a three-dimensional model of a virtual space, an image to be displayed on a display apparatus disposed in the real space. The apparatus generates the image in which an influence exerted by light from the light source on the virtual space has been affected.