2D Video Generation Using Real Camera Backgrounds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 2D delivery methods face challenges in producing realistic video backgrounds at low cost, as 3D CG videos require significant time and resources, and combining them with volumetric capture foregrounds results in unrealistic backgrounds that lack a live feeling.
Innovation Solution
An image processing system that generates 2D videos by acquiring camera positional information from a different location and synthesizes them with 3D models, using real camera data for backgrounds to create realistic and cost-effective 2D delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If 3D CG video is used as background in 2D delivery, then the background can be generated, but it takes a long time to produce and generates large cost
Solution Approach 1:
The patent uses real camera imaging to capture the background scene directly, creating a realistic background copy instead of generating it through time-consuming 3D CG rendering. This copying approach from reality eliminates the need for lengthy computer graphics production while maintaining visual authenticity.
Solution Approach 2:
The patent employs standard cameras and real-world scene capture instead of expensive, time-intensive 3D CG production pipelines. By using readily available imaging equipment and actual environments, the system achieves cost-effective background generation without requiring specialized rendering resources.
2Ease of manufacture
If 3D CG video is used as background, then background generation is possible, but the background lacks realism compared to the realistic foreground from volumetric capture
Solution Approach 1:
The system captures real-world backgrounds using cameras, creating authentic visual copies of actual scenes. This direct copying from reality ensures that the background maintains the same level of realism as the volumetric capture foreground, eliminating the artificial appearance of CG-generated environments.
Solution Approach 2:
The patent transitions from CG parameter-based background generation to real-world optical capture. By changing the fundamental parameter from computational rendering to physical imaging, the system achieves consistent realism across both foreground and background elements.
3Ease of manufacture
If pre-created 3D CG background video is synthesized, then background is available, but it looks like a pre-recorded video and lacks live feeling
Solution Approach 1:
The system uses real-time camera imaging to capture backgrounds dynamically as they occur, rather than using static pre-rendered CG sequences. This dynamic capture approach allows the background to reflect actual live conditions and movements, creating an authentic live broadcast feeling.
Solution Approach 2:
By continuously copying real-world scenes through camera imaging during the broadcast, the system maintains an up-to-date, authentic background that reflects actual live conditions rather than predetermined CG animations, thereby preserving the live broadcast atmosphere.
Data Source
AI summary
The present disclosure relates to an image processing apparatus and an image processing system capable of realizing 2D delivery capable of producing a realistic feeling at low cost. The image processing apparatus includes a 2D video generation unit that acquires, as virtual viewpoint information, camera positional information of a camera that performs imaging at a first place, and generates a 2D video obtained by viewing a 3D model of a person generated by performing imaging at a second place different from the first place from a viewpoint of the camera. The technique of the present disclosure can be applied to, for example, an image processing system that delivers a 2D video using volumetric capture.