Virtual Viewpoint Image Generation via Multi-Gaze Camera Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images from multiple camera views suffer from reduced image quality and accuracy, particularly when capturing players away from the gaze point, leading to blurred outlines and impaired realistic sensation.
Innovation Solution
The proposed system employs a network of cameras and image processing units to capture and process images from multiple directions, using 3D model data and foreground images to generate high-quality virtual viewpoint images, ensuring accurate modeling and focus across the entire scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If cameras are arranged to capture the entire stadium landscape with the center as the gaze point, then the coverage area is improved, but the image quality and resolution of players away from the center deteriorate
Solution Approach 1:
The patent divides the stadium into multiple capturing areas, each with its own gaze point and camera arrangement. Instead of using a single central gaze point to capture the entire stadium, the system segments the coverage into multiple regions (e.g., first capturing area with first gaze point, second capturing area with second gaze point). Each segment uses cameras optimally positioned for that specific region, ensuring high image quality for players in each segment while collectively covering the entire stadium.
Solution Approach 2:
The patent introduces multiple gaze points at different spatial positions and orientations rather than relying on a single central gaze point. By distributing gaze points across different locations (e.g., different sides of the stadium) and orientations, the system captures players from multiple dimensional perspectives, ensuring that players anywhere in the stadium are always within optimal capture range of at least one gaze point.
2Manufacturing precision
If cameras are positioned to focus on players near the gaze point, then the image quality of nearby players is improved, but the number of cameras capturing distant players is reduced
Solution Approach 1:
The patent assigns different groups of cameras to different capturing areas, with each group optimized for its specific region. The first group of cameras captures players in the first capturing area from the first gaze point, while the second group captures players in the second capturing area from the second gaze point. This segmentation ensures that each player is captured by the optimal number of cameras for their location, rather than relying on a single centralized camera system.
Solution Approach 2:
The patent implements local optimization by configuring camera parameters (such as focal length, aperture, and focus) specifically for each capturing area and gaze point. Each camera group is tuned to provide high-quality capture for its local region, with parameters adjusted based on the specific geometric relationship between that gaze point and the players in its capturing area. This local quality approach ensures that image quality is maximized for players wherever they are positioned in the stadium.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In an image processing system having a plurality of image capturing units, it aims to be able to generate a wide-area image from a high-quality virtual viewpoint. In order to achieve such an object, in the image processing system, there is provided a generating apparatus that generates a virtual viewpoint image on the basis of a plurality of captured images obtained by a plurality of image capturing devices, the plurality of image capturing devices being classified into a plurality of image capturing device groups including a first image capturing device group constituted by the plurality of image capturing devices pointed to a first gaze point and a second image capturing device group constituted by the plurality of image capturing devices pointed to a second gaze point different from the first gaze point, the generating apparatus comprising: a first obtaining unit configured to obtain first image data based on the captured image obtained by the image capturing device belonging to the first image capturing device group; a second obtaining unit configured to obtain second image data based on the captured image obtained by the image capturing device belonging to the second image capturing device group; a third obtaining unit configured to obtain information related to a position and a direction of a virtual viewpoint; and a generating unit configured to generate the virtual viewpoint image based on the first image data obtained by the first obtaining unit, the second image data obtained by the second obtaining unit, and the information related to the position and the direction of the virtual viewpoint obtained by the third obtaining unit.