Quasi-Uniform Spherical Grid for Neural Radiance Field Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual viewpoint image synthesis technologies, such as NeRF, face challenges in modeling large spaces efficiently due to high storage requirements and slow learning/rendering times, especially when using Cartesian coordinate systems, which are inefficient for large spaces and difficult for average users to implement.
Innovation Solution
The use of a quasi-uniform spherical coordinate system for constructing a neural network radiance field model, combining a Yin grid and a Yang grid with an exponentially increasing grid structure in the radial direction, allows for efficient storage and rendering of large spaces by using a small grid for nearby objects and a large grid for distant regions, optimizing environment lighting and reducing singular point issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a Cartesian coordinate system grid is used to model large spaces, then uniform resolution can be achieved, but storage requirements increase to O(n³) and rendering becomes inefficient
Solution Approach 1:
The patent transforms the coordinate system from Cartesian to spherical coordinates, fundamentally changing the parameterization of space. This allows the grid structure to adapt to the radial nature of the scene, using fewer bins in distant regions and more bins in near regions, thereby reducing overall storage requirements while maintaining perceptually uniform resolution
Solution Approach 2:
The patent introduces a radial dimension to the grid structure by using spherical coordinates (rho, theta, phi). This dimensional change allows the space to be partitioned differently, with the radial component (rho) providing exponential spacing that efficiently covers large spaces without requiring uniform fine-grained discretization throughout the entire volume
2Device complexity
If a Cartesian coordinate system grid is used to model large spaces, then simple grid structure can be maintained, but rendering time increases due to insufficient grids in near regions and excessive grids in far regions
Solution Approach 1:
By changing from linear Cartesian coordinates to spherical coordinates with exponential radial spacing, the patent creates a grid that naturally allocates more resolution to near regions and less to far regions. This parameter transformation makes the rendering process more efficient by concentrating computational resources where they are most needed
Solution Approach 2:
The patent creates a dynamic grid allocation strategy where the number of bins in each region is adapted to the importance and distance from the camera. Regions closer to the camera have exponentially more bins, allowing the system to dynamically prioritize rendering effort in visually critical areas
3Adaptability or versatility
If a large MLP is used to model the entire space in NeRF, then comprehensive scene coverage is achieved, but optimization time increases to 12 hours to 2 days
Solution Approach 1:
The patent segments the continuous space into discrete spherical bins organized by radial distance, angular elevation, and azimuthal angle. This segmentation allows the radiance field to be represented as a collection of localized bin values rather than a single global MLP, significantly reducing the number of parameters and optimization time while maintaining comprehensive scene coverage
Solution Approach 2:
The patent introduces an explicit spherical bin grid structure as an intermediary between the input coordinates and the radiance field representation. This intermediary structure organizes the scene into manageable discrete units, replacing the need for a large monolithic MLP and enabling faster optimization
Data Source
AI summary
An image synthesis device includes a memory storing a virtual viewpoint image synthesis program, and a processor configured to execute the program. The program receives images of multiple viewpoints obtained by filming around a user, position information on the user, and information on a direction in which scene is viewed, estimates color and density by inputting the position information of the user and the direction in which the object is viewed to a neural radiance field model constructed by using the images of the multiple viewpoints, and synthesizes images of a virtual viewpoint by performing volume rendering by using estimated color and density, the neural network radiance field model is constructed based on a quasi-uniform spherical coordinate system, and the quasi-uniform spherical coordinate system includes a sum of a Yin grid and a Yang grid and has a grid structure that increases exponentially in a radial direction.