3D Shape Estimation for Virtual Viewpoint Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating virtual viewpoint images from multiple camera angles often result in low shape accuracy, particularly when camera positions are close to one goal in sports events, leading to uncomfortable viewing experiences due to inaccurate object representation.
Innovation Solution
An image processing apparatus that sets a shape estimation region based on the gaze point and camera information, estimates the three-dimensional shape of objects within this region, and generates virtual viewpoint images using CPU-controlled units for camera information obtaining, shape estimation, and virtual image generation, ensuring accurate object representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If cameras are positioned close to one goal to capture specific game areas, then the coverage of that specific area is improved, but the shape estimation accuracy of objects in other areas deteriorates
Solution Approach 1:
The patent divides the image processing into multiple passes: first generating a preliminary 3D shape using all cameras, then identifying regions with low shape accuracy, and finally performing targeted shape correction only in those specific regions using appropriate camera data. This segmentation approach allows efficient processing while maintaining accuracy in critical areas.
Solution Approach 2:
The patent applies different processing quality levels to different regions of the image. High-shape-accuracy regions are processed with standard methods, while low-shape-accuracy regions undergo enhanced shape correction using data from cameras that have better viewing angles for those specific areas. This local quality adjustment ensures overall accuracy without uniformly increasing processing complexity.
2Reliability
If multiple cameras are used to capture objects from different directions, then the completeness of object capture is improved, but the complexity of the system increases
Solution Approach 1:
The patent combines data from multiple cameras through a unified 3D shape estimation process. By merging information from all camera viewpoints into a single coherent 3D model, the system achieves complete object capture while managing complexity through integrated processing rather than handling each camera independently.
Solution Approach 2:
The patent introduces a shape accuracy evaluation unit and a shape correction unit as intermediary components between the multiple cameras and the final virtual viewpoint image generation. These intermediaries manage the complexity by systematically evaluating and correcting shape accuracy, allowing the system to handle multiple camera inputs efficiently.
3Measurement precision
If shape estimation is performed for the entire image, then the overall shape accuracy is improved, but the processing time increases
Solution Approach 1:
The patent performs shape estimation on the entire image initially, then identifies only the regions with insufficient shape accuracy for correction. This partial action approach (correcting only where needed rather than re-processing everything) maintains overall shape accuracy while significantly reducing the time required compared to uniform full-image re-processing.
Solution Approach 2:
The patent performs a preliminary shape estimation for the entire image before identifying regions that need correction. This preliminary action establishes a baseline 3D model that can be quickly refined in specific areas, avoiding the need for time-consuming full-image re-processing while maintaining overall accuracy.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
An image processing apparatus (2) comprising: image obtaining means (100) for obtaining images based on capturing by a plurality of image capturing apparatuses; position obtaining means (125) for obtaining information representing a predetermined position to which the plurality of image capturing apparatuses are directed; region setting means (130) for setting, based on the information obtained by the position obtaining means, a region to estimate a three-dimensional shape of an object; and estimation means (140) for estimating, in the region set by the region setting means, the three-dimensional shape of the object based on the images obtained by the image obtaining means.