Virtual Viewpoint Image Generation with Selective Camera Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of virtual viewpoint images from multi-viewpoint video images is hindered by increased transfer time and shape estimation processing time, particularly in wide-field scenarios like soccer matches, which limits the real-time performance of live sports broadcasts.
Innovation Solution
A system comprising a first generation unit for creating a virtual viewpoint image based on captured images from multiple cameras, a determination unit to assess the need for higher image quality, and a second generation unit to generate high-quality virtual viewpoint images using data from specific camera groups, optimizing data transfer and processing by adjusting camera settings and using multi-viewpoint video images data from different camera groups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If shape estimation processing is performed for the entire wide field, then complete coverage of the field is achieved, but transfer time and processing time increase
Solution Approach 1:
The field is divided into multiple regions, each captured by different camera groups. The determination unit selectively determines which regions require high-quality virtual viewpoint images, and only transfers and processes data from those specific regions, rather than processing the entire field uniformly.
Solution Approach 2:
Different regions of the field receive different levels of processing quality. The determination unit identifies regions where high-quality virtual viewpoint images are necessary (e.g., regions containing important objects or actions) and applies enhanced processing only to those local areas, while using lower-quality processing for other regions.
2Manufacturing precision
If high-resolution video images are transferred and processed, then image quality improves, but transfer time increases
Solution Approach 1:
Instead of transferring and processing all high-resolution video images from all cameras, the system performs partial action by selectively transferring and processing only the necessary portions (specific camera groups and regions) where high image quality is required, as determined by the determination unit.
Solution Approach 2:
High image quality processing is applied locally only to specific regions and camera groups identified as necessary, rather than uniformly across the entire field, thereby reducing overall data transfer time while maintaining quality where needed.
3Area of stationary object
If multiple camera groups are used to capture the entire field, then coverage is improved, but data transfer and processing complexity increases
Solution Approach 1:
The system dynamically adjusts which camera groups are activated and used for processing based on real-time determination of what is necessary. The determination unit dynamically selects from available camera groups only those needed for the current scene, reducing complexity adaptively rather than maintaining fixed complexity for full coverage.
Solution Approach 2:
Multiple camera groups are designed with universal functionality to capture different regions of the field. The determination unit selectively combines data from these multi-functional camera groups based on current needs, allowing the system to achieve full field coverage capability while reducing actual processing complexity by using only the necessary subset.
Data Source
AI summary
A virtual viewpoint image generation apparatus including: a first generation unit configured to generate, based on a plurality of captured images obtained by a plurality of first cameras, a first virtual viewpoint image in accordance with a position and direction of a virtual viewpoint; a determination unit configured to determine, in accordance with evaluation results of the first virtual viewpoint image, whether or not to generate a second virtual viewpoint image whose image quality is higher than that of the first virtual viewpoint image based on one or a plurality of captured images obtained by one or a plurality of second cameras; and a second generation unit configured to generate the second virtual viewpoint image whose image quality is higher than that of the first virtual viewpoint image in accordance with determination by the determination unit.