Virtual Viewpoint Image Generation Using 3D Background Data Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of high-quality virtual viewpoint images is hindered by increased processing loads, particularly when attempting to faithfully reproduce large-scale three-dimensional backgrounds in dynamic environments like sports stadiums, where unnecessary objects are included in pre-captured 3D data or essential objects are missing.
Innovation Solution
A system that generates simulation images based on three-dimensional shape data, detects differences between simulation and actual images, and updates the background data accordingly, using a network of cameras, a server, and a control device to separate foreground and background images, and update the 3D data in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-quality virtual viewpoint images are generated by faithfully reproducing three-dimensional background data, then image quality is improved, but processing load increases
Solution Approach 1:
The patent divides the complex task of 3D background reproduction into separate stages: initial 3D data creation, simulation image generation, difference detection, and selective updating. This segmentation allows processing to be focused only on areas requiring improvement rather than reprocessing entire backgrounds, reducing overall processing load while maintaining accuracy.
Solution Approach 2:
The system performs partial updating of 3D background data by only processing regions where differences between simulation and captured images are detected. Instead of completely regenerating all background data, it applies updates only where necessary, reducing processing requirements while achieving high-quality results.
2Speed
If pre-captured 3D data is used for virtual viewpoint generation, then processing speed is improved, but accuracy deteriorates due to missing or unnecessary objects
Solution Approach 1:
The system implements a feedback mechanism where captured images are compared against simulation images generated from pre-captured 3D data. The detected differences are fed back to update and refine the 3D background data, ensuring it remains accurate with current event configurations while preserving the efficiency of using pre-captured data.
Solution Approach 2:
The patent performs preliminary 3D data capture and simulation image generation before actual event coverage. This allows the system to prepare optimized 3D backgrounds in advance, then quickly refine them using real-time comparisons during events, maintaining both speed and accuracy.
Data Source
AI summary
A system that generates a virtual viewpoint image from multiple viewpoint images captured from different camera viewpoints uses background 3D data representing a three-dimensional shape of a background in an image capturing scene. This background 3D data is generated by using a simulation image representing a view from the camera viewpoint. Then, a difference between the simulation image and an image acquired by performing test image capturing from the same camera viewpoint as the camera viewpoint is detected and the background 3D data is updated based on detection results.