3D Graphic Insertion Using Virtual Camera Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods fail to accurately insert 3D graphics into live stereoscopic video streams, breaking the illusion of graphics being part of the real scene due to disparities in image formation characteristics between virtual and actual camera models.
Innovation Solution
A method and system that generate graphics for both channels of stereoscopic content using virtual camera models representing the image formation process of the stereoscopic rig, including camera characteristics like toe-in and inter-optical separation, with optional 2D spatial remapping to correct for optical distortions and sensor misalignment, ensuring parallax offsets are applied uniformly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fixed parallax of graphic elements is used, then graphic insertion is simplified, but the perceived location of graphics does not correspond with the perceived location of the scene
Solution Approach 1:
The patent applies dynamics by transitioning from fixed parallax to variable parallax that adapts to different scene depths. The system dynamically calculates parallax values based on the depth of graphical elements relative to the camera position and orientation, allowing graphics to be accurately placed at various depths in the stereoscopic scene rather than at a single fixed parallax level.
Solution Approach 2:
The patent changes the parallax parameter from a fixed value to a variable that depends on scene depth and camera parameters. By adjusting the parallax parameter based on the virtual camera model and the desired depth position of graphics, the system achieves accurate perceived location correspondence between graphics and the real scene while maintaining operational simplicity through automated parameter calculation.
2Ease of manufacture
If virtual camera model does not match actual camera characteristics, then graphic generation is simpler, but disparity between graphics and scene breaks the illusion
Solution Approach 1:
The patent applies copying by creating a virtual camera model that replicates the image formation characteristics of the actual stereoscopic camera rig. The virtual camera copies the essential parameters including position, orientation, inter-optical separation, and toe-in angle, allowing accurate generation of graphics that correspond to the real scene while simplifying the complex task of matching all camera characteristics by focusing on the most critical parameters.
3Adaptability or versatility
If parallax offsets are not applied uniformly, then processing flexibility is increased, but the illusion of graphics being part of the scene is broken
Solution Approach 1:
The patent applies local quality by applying uniform parallax offsets specifically to computer-generated graphical elements while leaving the captured video frames unchanged. This localized application ensures that graphics and video share the same perceived depth relationship, maintaining the illusion of graphics being part of the scene, while still allowing flexibility in the overall processing pipeline for different graphic types and positions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate and immersive insertion of graphics into live 3D video streams, maintaining the illusion of graphics being at defined positions in 3D space, enhancing viewer experience by correcting for image formation discrepancies and ensuring high-quality stereoscopic content.
Implementation Method 1
obtaining two or more actual video stream channels of images of the real situs from two or more laterally-displaced digital cameras directed at the real situs
Implementation Method 2
combining the actual-virtual video stream channels and, using the inter-camera positional data, horizontally translating the same with respect to one another based upon desired parallax offset to form a 3D blended actual-virtual image combined video stream
Data Source
AI summary
Systems and methods for the insertion of graphics into stereoscopic live action 3D video using image models taking into account positional data from the stereoscopic camera rig.