Virtual Viewpoint Image Reconstruction Using Weighted Multi-Camera Blending
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing techniques for reconstructing images from multiple viewpoints often result in unnatural color changes due to differences in lighting and capturing conditions, leading to incongruity between blended regions.
Innovation Solution
An image processing apparatus and method that determines color information of an object from a virtual viewpoint by combining color information from selected captured images based on the position, orientation, and capturing direction of image capturing apparatuses, using weights corresponding to the image capturing direction and object position within the field of view, to reduce color inconsistencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If captured images from multiple image capturing apparatuses are blended to reconstruct an image from a virtual viewpoint, then the visual completeness and coverage of the reconstructed image is improved, but color inconsistencies and unnatural color changes occur in the blended regions
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the mixing ratios of captured images based on multiple parameters including the distance between the virtual viewpoint and each image capturing apparatus, the shooting direction alignment, and the position of objects within fields of view. This allows the system to select and blend images with appropriate weightings that minimize color inconsistencies while maintaining complete visual coverage of the reconstructed image.
Solution Approach 2:
The patent implements local quality by applying different mixing ratios to different captured images based on their specific characteristics and relevance to the virtual viewpoint. Instead of uniform blending, each captured image is weighted according to its positional relationship with the virtual viewpoint and the object's location within its field of view, ensuring that regions closer to the virtual viewpoint contribute more to the final reconstructed image while maintaining color consistency.
2Manufacturing precision
If captured images from apparatuses positioned near the virtual viewpoint or with line-of-sight close to the virtual viewpoint direction are selected and blended with higher mixing ratios, then the natural appearance of the reconstructed image is improved, but the complexity of selecting and weighting multiple captured images increases
Solution Approach 1:
The patent manages complexity by systematically evaluating multiple parameters (distance to virtual viewpoint, shooting direction alignment, object position in field of view) and automatically calculating optimal mixing ratios based on these parameters. This structured approach to parameter-based selection and weighting simplifies the complex process of choosing and blending multiple captured images while maintaining natural appearance in the reconstructed image.
3Adaptability or versatility
If multiple captured images are blended to reconstruct an image from an arbitrary virtual viewpoint, then the versatility of the image processing system is improved, but color inconsistencies due to different lighting and capturing conditions worsen
Solution Approach 1:
The patent resolves the contradiction between versatility and color consistency by dynamically adjusting mixing ratios based on the specific geometric and optical parameters of each captured image relative to the virtual viewpoint. The system considers distance, shooting direction, and object position to weight images appropriately, allowing reconstruction from any virtual viewpoint while compensating for color inconsistencies caused by different lighting and capturing conditions through intelligent blending.
Data Source
AI summary
There is provided with an image processing apparatus. A determination unit determines color information of an object present in an object image from a virtual viewpoint. The color information of the object is determined by combining color information of the object in a captured image obtained by one or more image capturing apparatus selected based on position and orientation information of the virtual viewpoint, position and orientation information of the image capturing apparatuses, and position information of the object, by using a weight corresponding to an image capturing direction of the one or more image capturing apparatus and a position of the object in a field of view of the one or more image capturing apparatus.