Large-Scene Semantic Light Field Reconstruction from Sparse Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for large-scene light field reconstruction require dense observation, leading to reduced completeness and precision due to information loss, especially in sparse views, and fail to effectively model complex geometric surfaces and retain geometric details.
Innovation Solution
A method and system that decompose the large scene into semantic primitives, using multi-layer perceptron networks to optimize geometric boundaries and local details, enabling high-completeness and high-precision reconstruction with sparse views by integrating semantic information and visibility estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dense observation is used for light field reconstruction, then reconstruction precision is improved, but observation cost and data requirement increase
Solution Approach 1:
The patent segments the continuous light field space into discrete voxel grids, allowing sparse sampling points to represent and reconstruct the entire light field through network modeling. This segmentation enables high-precision reconstruction with minimal observation points by distributing reconstruction tasks across multiple grid units.
Solution Approach 2:
The patent introduces neural network models as intermediaries between sparse observation points and the final light field reconstruction. The network models process and interpolate data from limited sampling points to generate complete light field information, acting as a mediator that transforms sparse inputs into dense reconstructions.
2Quantity of substance
If sparse observation is used for light field reconstruction, then observation cost is reduced, but reconstruction completeness and precision deteriorate
Solution Approach 1:
The patent performs preliminary organization of sparse observation data into structured voxel grids before reconstruction. By pre-arranging sampling points into grid-based semantic primitives and preparing network models in advance, the system enables accurate reconstruction from sparse data without requiring dense observations during the actual reconstruction process.
Solution Approach 2:
The patent changes the parameter representation from continuous spatial coordinates to discrete voxel grid indices. This parameter transformation allows the system to work efficiently with sparse observations by mapping limited sampling points to specific grid locations, where network models can interpolate and reconstruct complete light field information.
3Measurement precision
If complex geometric surfaces are modeled in detail, then reconstruction precision is improved, but computational complexity increases
Solution Approach 1:
The patent transitions from modeling complex geometric surfaces in continuous 3D space to representing them in a discrete 4D voxel grid space (x, y, z, time/observation). This dimensional transformation simplifies the representation of complex surfaces by using grid-based semantic primitives that can be processed efficiently by network models while maintaining reconstruction precision.
Data Source
AI summary
A method includes: acquiring a semantic primitive set of a multi-view image set; acquiring a coordinate offset by inputting coordinate information and a feature vector corresponding to a first grid sampling point of the semantic primitive set into a first network model, and acquiring a second grid of the semantic primitive set based on the coordinate offset and geometric attribute information of the semantic primitive set; acquiring first feature information of a second grid sampling point by inputting coordinate information and a feature vector corresponding to the second grid sampling point, and an observation angle value into a second network model, and acquiring second feature information of the semantic primitive set based on the first feature information; and acquiring a light field reconstruction result of the multi-view image set based on an observation angle value of the semantic primitive set and third feature information extracted from the second feature information.