Virtual Viewpoint Image Visibility for Directivity Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating virtual viewpoint image data often result in reduced visibility of objects with high directivity light sources, such as LED displays, when captured from outside their viewing angle, leading to dark and less visible renderings in virtual images.
Innovation Solution
An image processing apparatus that acquires camera and object information to determine the priority of captured image data based on the viewing angle and orientation of objects, preferentially using image data from cameras closest to the object's normal direction to enhance visibility in virtual viewpoint images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual viewpoint image data is generated using captured image data from cameras located outside the viewing angle of objects with high directivity surfaces, then the coverage of viewpoints is increased, but the visibility of the object in the virtual viewpoint image is reduced
Solution Approach 1:
The system pre-acquires viewing angle information for objects with high directivity surfaces and stores it in advance. When generating virtual viewpoint images, the system retrieves this pre-stored information to determine the appropriate captured image data, avoiding the need to calculate viewing angles in real-time and enabling efficient selection of suitable camera data that ensures object visibility.
Solution Approach 2:
The system changes the selection criterion for captured image data from simply proximity to the virtual viewpoint to a composite parameter that includes the angle between the camera's optical axis and the object's normal direction. This parameter change allows the system to select image data that maintains object visibility while still providing viewpoint coverage.
2Loss of information
If captured image data from all available cameras is used for rendering, then the completeness of the virtual viewpoint image is improved, but the accuracy of object appearance is reduced due to using images taken from inappropriate angles
Solution Approach 1:
The system applies different selection criteria to different objects in the scene. For objects with high directivity surfaces, it uses the viewing angle criterion (selecting images where the camera is within an appropriate angle of the object's normal direction). For other objects without such constraints, it can use standard proximity-based selection. This local quality approach ensures each object is rendered with the most appropriate image data for its specific characteristics.
Solution Approach 2:
The system dynamically adjusts the selection of captured image data based on the virtual viewpoint position and object characteristics. As the virtual viewpoint moves, the system recalculates which cameras provide appropriate viewing angles for each object, enabling the rendering to adapt to different viewing conditions while maintaining object appearance accuracy.
Data Source
AI summary
The technique of this disclosure suppresses a reduction in visibility of a predetermined object in virtual viewpoint image data. An image processing apparatus includes: an image capturing information acquisition unit configured to acquire image capturing information indicating a position and orientation of each of a plurality of image capturing apparatuses; an object information acquisition unit configured to acquire object information indicating a position and orientation of an object to be captured by the image capturing apparatuses, the object having a specific viewing angle; and a determination unit configured to determine, based on the acquired image capturing information and the position and orientation of the object indicated by the acquired object information, an image to be used for generating a virtual viewpoint image according to a position and orientation of a virtual viewpoint among a plurality of images based on capturing by the image capturing apparatuses.