Virtual Viewpoint Image Generation Excluding Non-Target Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual viewpoint image generation techniques often result in increased processing load and data size, and may not accurately reflect the creator's intention, as they include objects not intended to be displayed within the virtual viewpoint.
Innovation Solution
An image processing apparatus that acquires viewpoint information and determines a three-dimensional area for a specific object, generating a virtual viewpoint image by excluding objects outside this area, thereby controlling what is displayed based on the virtual viewpoint's field of view and intended drawing area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all objects within the image capturing range of the virtual camera are drawn as they are, then the virtual viewpoint image is generated, but the processing load increases and the data size increases
Solution Approach 1:
The patent extracts only the necessary objects (those within the three-dimensional area defined by the specific object) from the complete image capturing range and processes only these extracted objects for virtual viewpoint image generation. This reduces the processing load by focusing computation only on relevant objects rather than all objects in the scene.
Solution Approach 2:
The patent segments the image capturing area into a three-dimensional area defined by the specific object and the rest. By dividing the scene into relevant and irrelevant portions, the system can process only the relevant segment (objects within the three-dimensional area) while excluding unnecessary objects, thus reducing overall processing load.
2Reliability
If all objects within the image capturing range of the virtual camera are drawn as they are, then the virtual viewpoint image is generated, but the data size of the virtual viewpoint image increases
Solution Approach 1:
The patent extracts only the necessary object data (objects within the three-dimensional area) from the complete image data and uses only this extracted data for virtual viewpoint image generation. This reduces the data size by excluding unnecessary object information while maintaining the accuracy of representing the creator's intention for the specific object.
Solution Approach 2:
The patent segments the image data into a three-dimensional area defined by the specific object and the rest. By dividing the data into relevant and irrelevant portions, the system can store and process only the relevant segment, reducing the overall data size while maintaining necessary information accuracy.
3Productivity
If all objects within the image capturing range of the virtual camera are drawn as they are, then the virtual viewpoint image is generated, but the virtual viewpoint image does not accurately reflect the creator's intention
Solution Approach 1:
The patent applies local quality by differentiating the treatment of objects based on their spatial relationship with the specific object. Objects within the three-dimensional area defined by the specific object are processed and included in the virtual viewpoint image, while objects outside this area are excluded. This localized processing ensures the image accurately reflects the creator's intention for the specific object while improving processing efficiency.
4Productivity
If a three-dimensional area is determined for a specific object and objects outside this area are excluded, then the processing load and data size are reduced, but the field of view constraints must be precisely controlled
Solution Approach 1:
The patent performs preliminary action by determining the three-dimensional area defined by the specific object before generating the virtual viewpoint image. By pre-calculating this three-dimensional area, the system can subsequently and efficiently exclude objects outside this area during image generation, reducing processing load while managing the complexity of area determination through advance preparation.
Data Source
AI summary
The image processing apparatus of the present invention is an image processing apparatus that generates a virtual viewpoint image based on image data obtained by capturing an image capturing area from a plurality of directions by a plurality of cameras, the image processing apparatus including: an acquisition unit configured to acquire viewpoint information of a virtual viewpoint; an area determination unit configured to determine a three-dimensional area in accordance with a position and a size of a specific object within the image capturing area; and a generation unit configured to generate the virtual viewpoint image in accordance with the virtual viewpoint indicated by the viewpoint information based on determination by the area determination unit such that an object within a field of view in accordance with the virtual viewpoint and not included in the three-dimensional area determined by the area determination unit is not displayed in the virtual viewpoint image.