Real-Time 3D Video Composition Using Depth Buffer Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for video composition, such as combining a live stream with pre-calculated 2D or 3D content, cannot effectively map a live stream onto 3D geometry or manage occlusion, limiting the quality and complexity of rendered effects.
Innovation Solution
A method for live construction of a video sequence that involves pre-calculating data for a 3D environment and depth information, live mapping a live video stream onto a 3D object, and composing the sequence using depth information to manage occlusion, allowing for high-quality 3D video composition with dynamic camera movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a transparency key method is used to combine video streams, then the combination process is simple, but the ability to map live stream on 3D geometry and manage occlusion is lost
Solution Approach 1:
The patent transitions from 2D layer-based composition to 3D spatial composition by introducing depth buffers and Z-coordinate information. Each pixel is associated with a depth value that enables three-dimensional occlusion testing, allowing live video streams to be properly integrated with 3D geometry while maintaining composition simplicity.
2Manufacturing precision
If pre-calculated video is used for complex effects, then rendering quality is high, but real-time composition capability is reduced
Solution Approach 1:
The patent pre-calculates and stores depth information (depth buffers) for 3D environments in advance, while keeping the actual video composition process lightweight and real-time. The pre-calculated depth data enables fast occlusion testing during live composition without requiring complex real-time rendering calculations.
Solution Approach 2:
The patent separates the composition process into independent components: pre-calculated depth buffers for 3D environments and real-time processed live video streams. This segmentation allows each component to be optimized independently, maintaining high rendering quality while achieving real-time composition performance.
3Productivity
If 2D layer combination is used, then the composition process is fast, but occlusion management and 3D mapping capability are limited
Solution Approach 1:
The patent introduces depth buffers as an intermediary data structure that mediates between 2D video layers and 3D spatial relationships. The depth buffer stores Z-coordinate information for each pixel, enabling accurate occlusion management while maintaining the efficiency of layer-based composition operations.
Data Source
AI summary
A method for real time construction of a video sequence comprising a modelled 3D object is provided. The method comprises pre-calculating data representative of a first image of a three-dimensional environment and a first associated depth information. A live calculating of data representative of a second image representing the modelled object on which is mapped a current image of a live video stream, and a second depth information associated with the second image is performed. The sequence is composed by combining the first image and the second image according to the first and second depth information.