Object-Based Radiance Field Learning for Multi-Object Scene Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for estimating radiance fields in a target space fail to consider the positional relationship between objects, leading to difficulties in dividing the space into convex polyhedron regions, resulting in combined volume densities for multiple objects within the same region, which hinders accurate estimation of individual object densities.
Innovation Solution
An information processing apparatus that sets learning regions based on object positions, associates three-dimensional space models with these regions, and performs learning to estimate radiance fields separately for each object, using multi-viewpoint images and camera parameters to generate accurate volume densities for each object.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the space is divided into convex polyhedron regions based on volume density distribution, then the efficiency of learning and image generation is improved, but the ability to separate multiple objects is deteriorated
Solution Approach 1:
The patent divides the scene into multiple convex polyhedron regions based on object positional information obtained from multi-viewpoint images. Each region is assigned to a specific object, allowing separate learning of radiance fields for each object. This segmentation enables both efficient processing through regional division and accurate object separation by ensuring each object occupies its own dedicated region.
2Device complexity
If multiple objects are included in the same divided region, then the complexity of regional division is reduced, but the accuracy of individual object volume density estimation is deteriorated
Solution Approach 1:
The patent performs preliminary extraction of object positional information from multi-viewpoint images before dividing the space into regions. By obtaining bounding boxes or spatial coordinates of objects in advance, the system can pre-arrange convex polyhedron regions to match object positions, ensuring each object is isolated in its own region before the radiance field learning process begins.
3Ease of manufacture
If regional division is made without considering object positions, then the ease of region generation is improved, but the reliability of object-specific radiance field estimation is deteriorated
Solution Approach 1:
The patent applies different division strategies to different regions based on local object characteristics. By using object positional information, the system creates convex polyhedron regions that are locally optimized for each object's spatial location and shape. This local adaptation ensures that each region's boundaries are tailored to contain specific objects, improving the reliability of object-specific radiance field estimates while maintaining systematic generation through automated object detection.
Data Source
AI summary
Radiance fields are estimated separately for each object for a scene in which a plurality of objects are present. An information processing apparatus obtains data on a plurality of captured images obtained through image capturing from a plurality of viewpoints, a camera parameter in image capturing of each of the plurality of captured images, and object information indicating a position of each of a plurality of objects included as representations in the captured images, sets a plurality of learning regions based on the object information, associates a three-dimensional space model with each of the plurality of learning regions based on a number of objects included in each of the plurality of learning regions, and performs learning of the three-dimensional space model associated with each of the plurality of learning regions based on the data on the plurality of captured images, the camera parameter, and the object information.