Virtual Viewpoint Generation via 3D Shape and Color Data Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images from multiple camera views require large data amounts, leading to inefficiencies in processing and storage.
Innovation Solution
An image processing system that generates three-dimensional shape data and color information from multiple images, storing these as material data for virtual viewpoint images, reducing the need for extensive image data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If images from a plurality of cameras are used to generate virtual viewpoint images, then image quality and viewing angles are improved, but data amount becomes very large
Solution Approach 1:
The patent extracts only the essential geometric information (3D shape data) and visual properties (color information) from the original images, separating these key elements from the complete image data. This extraction process retains the necessary information for virtual viewpoint generation while eliminating redundant data, thereby reducing the overall data amount while maintaining adaptability for various viewing angles.
Solution Approach 2:
The patent segments the image data into distinct components: 3D shape data representing the geometric structure and color information representing visual properties. This segmentation allows each component to be processed and stored independently, enabling efficient reconstruction of virtual viewpoints without requiring storage of complete high-resolution images from all cameras.
2Manufacturing precision
If complete image data from multiple cameras is stored, then image quality is maintained, but storage capacity is consumed
Solution Approach 1:
Instead of storing complete original images, the patent creates simplified representations (3D shape data and color information) that serve as copies containing only the essential visual characteristics. These copied data structures preserve the necessary image quality for virtual viewpoint generation while occupying significantly less storage capacity than the original multi-camera image sets.
3Quantity of substance
If three-dimensional shape data and color information are generated and stored, then data amount is reduced, but processing complexity increases
Solution Approach 1:
The patent performs preliminary processing to convert raw images into 3D shape data and color information before virtual viewpoint generation. This preliminary action pre-computes the geometric and visual characteristics, so that subsequent virtual viewpoint rendering can proceed more efficiently by simply reconstructing views from these preprocessed data structures rather than processing complete images each time.
Data Source
AI summary
In an image processing apparatus, three-dimensional shape data indicating a shape of an object is generated based on at least one of a plurality of images captured from different directions by a plurality image capturing devices. Color information indicating a color of each element of the three-dimensional shape data is generated based on the generated three-dimensional shape data and the plurality of images. The three-dimensional shape data and the color information are associated with each other and written to a storage unit as material data for generating a virtual viewpoint image corresponding to virtual viewpoint information indicating a virtual viewpoint.