Virtual Viewpoint Image Generation Selective Camera Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing three-dimensional modeling techniques face challenges in reducing overall computation while maintaining accuracy, especially when dealing with multiple objects, as the accuracy of three-dimensional models changes uniformly across objects, and the demand for reducing computation amount is not adequately met.
Innovation Solution
An image processing system that selectively uses a limited number of imaging units for generating three-dimensional models based on whether an object is a target or non-target object, employing the space carving method within the Visual Hull framework to reduce computation while maintaining accuracy for target objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a higher number of cameras are used for three-dimensional modeling, then the accuracy of the three-dimensional model is improved, but the processing time increases
Solution Approach 1:
The patent segments the three-dimensional modeling process into two distinct phases: Visual Hull processing for obtaining the general shape, and feature amount matching for obtaining details. This segmentation allows the system to use all cameras for the initial coarse modeling (maintaining accuracy) while using fewer cameras for the detailed processing (reducing time), thereby resolving the contradiction between model accuracy and processing speed
Solution Approach 2:
The patent applies partial action by using all cameras for Visual Hull processing (excessive for final detail accuracy but necessary for general shape) and then using only selected cameras for feature amount matching (partial sufficiency for details). This staged approach ensures accuracy where most needed while reducing overall computation time
2Measurement precision
If a higher number of cameras are used for three-dimensional modeling, then the accuracy of the three-dimensional model is improved, but the computation amount increases
Solution Approach 1:
The patent divides the computation into two segments: Visual Hull computation using all cameras for general shape (lower computational complexity per camera), and feature amount matching computation using selected cameras for details (higher computational complexity but fewer cameras). This segmentation reduces the overall computation amount while maintaining accuracy by optimizing the camera selection for each computational stage
Solution Approach 2:
The system performs partial feature amount matching using only selected cameras rather than all cameras, which significantly reduces the computation amount required for detailed three-dimensional modeling while maintaining sufficient accuracy through the prior Visual Hull general shape establishment
3Measurement precision
If the same processing method is applied to all objects, then the accuracy changes uniformly across objects, but the ability to reduce overall computation amount is limited
Solution Approach 1:
The patent applies local quality by selecting different subsets of cameras for different objects based on their individual characteristics and positions. Instead of uniform processing, each object receives customized camera selection optimized for its specific three-dimensional modeling needs, thereby reducing overall computation amount while maintaining accuracy for each target object
4Measurement precision
If feature amount matching is used for all objects, then the accuracy of three-dimensional models is improved, but the processing time increases significantly
Solution Approach 1:
The patent segments the processing approach by applying Visual Hull (faster, less accurate) for all objects to obtain general shapes, and then applying feature amount matching (slower, more accurate) only to selected cameras for specific objects requiring higher accuracy. This segmentation dramatically reduces overall processing time while maintaining necessary accuracy for target objects
Solution Approach 2:
The system applies feature amount matching partially rather than universally - using it only for objects where high accuracy is required and only with selected cameras. This partial application of the computationally intensive method significantly reduces processing time while maintaining accuracy where most needed
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
An image processing apparatus, which executes image processing for generating a virtual viewpoint image using a plurality of images captured by a plurality of imaging units that image an imaging space from different viewpoints, identifies a specific object among a plurality of objects inside the imaging space, and carries out image processing for generating the virtual viewpoint image on the plurality of objects inside the imaging space. The image processing apparatus executes the image processing on the identified specific object using images captured by more imaging units than in the image processing executed on other objects.