3D Background Video Compositing to Eliminate Moire and Ghosting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Capturing a background video displayed on a large display device results in unintended artifacts such as distortion and noise, particularly moire and ghost effects, in the captured video.
Innovation Solution
An information processing device synthesizes a captured video with a second video generated from rendered 3D background data, cutting out the object from the captured video and replacing the defective portions with the second video to eliminate artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a background video is displayed on a large display device and captured by a camera, then the performer and staff can visually understand the scene and determine performance quality, but various artifacts such as distortion, noise, and moire occur in the captured video
Solution Approach 1:
The patent creates a virtual copy of the background video through real-time rendering of 3D background data. Instead of capturing the actual displayed background video which contains artifacts, the system generates a synthetic copy that replicates the background scene without the moire and distortion issues, then composites it with the captured performer video.
Solution Approach 2:
The patent introduces an intermediary processing system that separates the background and foreground components. The camera captures both performer and background together, then the system extracts the performer video, generates a separate rendered background video, and composites them. This intermediary step eliminates artifacts by avoiding direct capture of the displayed background.
2Ease of manufacture
If green back shooting is used to produce video content, then background video synthesis is required, but the performer and staff cannot visually understand the scene during performance
Solution Approach 1:
The patent creates a virtual copy of the background video through real-time rendering of 3D background data. Instead of capturing the actual displayed background video which contains artifacts, the system generates a synthetic copy that replicates the background scene without the moire and distortion issues, then composites it with the captured performer video.
3Productivity
If a captured video is used directly without processing, then the workflow is simple, but artifacts such as moire and ghost effects remain in the final video content
Solution Approach 1:
The patent extracts the performer video from the captured video that contains both performer and background. By separating the foreground (performer) from the background, the system can replace the defective background portion with a cleanly rendered version while preserving the performer capture, thus eliminating artifacts without complex post-processing.
Solution Approach 2:
The patent segments the captured video into performer video and background video components. This segmentation allows independent processing of each component - the performer video is kept as captured while the background video is regenerated through rendering, enabling artifact removal while maintaining workflow efficiency.
Data Source
AI summary
An information processing device includes a video processing unit that generates a synthesized video by synthesizing an object video and a second video obtained by rendering second 3D background data, the object video being cut out from a captured video obtained by imaging an object with a display device as a background, the display device displaying a first video obtained by rendering first 3D background data.