Elastic Semantic Light Field Reconstruction for Large Scene Point Clouds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current light field reconstruction methods for large scenes suffer from low accuracy, integrity, and quality, particularly in self-supervised algorithms which rely heavily on photometric consistency and struggle with non-ideal Lambert surfaces and non-textured areas.
Innovation Solution
A method for large scene elastic semantic representation and self-supervised light field reconstruction, involving the acquisition of depth maps, normal vector maps, and confidence measure maps, followed by iterative training using a target elastic semantic reconstruction model that performs pixel propagation and fusion to generate high-quality scene point clouds without additional depth supervision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional photometric consistency-based light field reconstruction algorithms are used, then the reconstruction process can be performed without additional depth supervision, but the accuracy and quality of reconstruction are low, especially for non-ideal Lambert surfaces and non-textured areas
Solution Approach 1:
The patent introduces elastic semantic representation as an intermediary to bridge the gap between photometric consistency and geometric consistency. The semantic representation serves as a mediator that guides the pixel propagation process, enabling accurate depth reconstruction in challenging areas without requiring complex additional supervision mechanisms
Solution Approach 2:
The patent transforms the reconstruction approach by changing from direct photometric matching to semantic-guided pixel propagation. The key parameter change is the introduction of semantic features (color, texture, geometry) as propagation constraints, which fundamentally alters how depth information is reconstructed and improves accuracy for non-ideal surfaces
2Reliability
If elastic semantic representation with pixel propagation is used, then the integrity and quality of light field reconstruction are improved, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary elastic semantic representation learning before the pixel propagation process. By pre-computing the semantic features and establishing the elastic representation model in advance, the actual reconstruction process benefits from these prepared structures, reducing the computational burden during real-time processing
Solution Approach 2:
The patent segments the reconstruction process into distinct stages: elastic semantic representation learning, pixel propagation with semantic constraints, and depth map fusion. This segmentation allows each stage to be optimized independently and enables parallel processing where applicable, improving overall efficiency
3Measurement precision
If multiple depth maps from different angles are processed through elastic semantic reconstruction, then the accuracy of scene point cloud is improved, but the device complexity and computational resources required increase
Solution Approach 1:
The patent designs the elastic semantic reconstruction model to handle multiple depth maps from different angles using the same unified framework. The model performs multi-functional operations including semantic feature extraction, pixel propagation, and depth fusion within a single system, avoiding the need for separate processing pipelines for each angle view
Data Source
AI summary
A method for large scene elastic semantic representation and self-supervised light field reconstruction is provided. The method includes acquiring a first depth map set corresponding to a target scene, in which the first depth map set includes a first depth map corresponding to at least one 5 angle of view; inputting the first depth map set into a target elastic semantic reconstruction model to obtain a second depth map set, in which the second depth map set includes a second depth map corresponding to the at least one angle of view; and fusing the second depth map corresponding to the at least one angle of view to obtain a target scene point cloud corresponding to the target scene.