Image Processing Apparatus Learning Area Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating images corresponding to virtual viewpoints from multi-viewpoint images, such as NeRF and VaxNeRF, require extensive time for learning due to the large number of sampling points and pixels involved, especially in scenes with many objects distributed across a wide image capturing area.
Innovation Solution
The proposed image processing apparatus generates rough shape data using the visual hull method to define a learning area for each object, limiting the sampling points within this area and performing independent learning for each object's three-dimensional field, thereby reducing the amount of information to process and accelerating convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the visual hull method is used to limit sampling points within the three-dimensional shape, then the learning speed is increased, but the accuracy may deteriorate when objects are sparsely distributed in a wide image capturing area
Solution Approach 1:
The patent divides the image capturing area into multiple learning areas based on the positions of individual objects. Each learning area is defined by a bounding box that encloses only one object, allowing the system to process and learn from sparse object distributions more efficiently while maintaining accuracy for each individual object.
2Reliability
If sampling is performed over the entire image capturing area, then the completeness of learning is improved, but the time required for learning increases significantly
Solution Approach 1:
The patent extracts and isolates individual objects from the entire scene by creating separate learning areas for each object. This extraction allows the system to focus computational resources only on relevant regions containing objects, eliminating wasted computation on empty spaces while ensuring complete learning coverage of all objects in the scene.
3Adaptability or versatility
If the learning area is expanded to cover the entire scene, then all objects are captured, but the computational complexity increases
Solution Approach 1:
The patent segments the computational domain by creating separate learning areas for each object rather than processing the entire scene as one large volume. This segmentation reduces the computational complexity of each individual learning task while maintaining the ability to handle diverse scenes with multiple objects at various positions and scales.
Data Source
AI summary
To perform leaning for generating an image or the like corresponding to a virtual viewpoint from multi-viewpoint images at a higher speed. Shape data representing a three-dimensional shape of an object is generated based on a plurality of captured images obtained by a plurality of image capturing devices. Then, a learning area is set for each object based on the generated shape data and a three-dimensional field in accordance with a captured image is learned by taking the learning area set for each object as a target.