Manhattan Layout Estimation Using Geometric and Semantic Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently compressing and transmitting large volumes of 3D point cloud data for immersive media, leading to high data storage and transmission requirements.
Innovation Solution
The use of a V-PCC scheme that converts 3D point cloud frames into image-based representations, utilizing video coding techniques to encode geometry, occupancy, and texture, with metadata compression, to reduce data volume.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D point cloud data is used to represent immersive media scenes, then the quality and realism of the 3D representation is improved, but the data volume and transmission requirements increase significantly
Solution Approach 1:
The patent creates 2D image-based representations (copies) of the 3D point cloud data. Instead of transmitting the full 3D point cloud, the system encodes multiple 2D images that represent different views or aspects of the 3D scene, significantly reducing data volume while preserving the ability to reconstruct the 3D scene when needed.
Solution Approach 2:
The patent transforms 3D point cloud data into 2D image representations for compression and transmission. By converting the data from three-dimensional space to two-dimensional space, the system achieves more efficient encoding and transmission while maintaining the essential spatial relationships needed for 3D scene reconstruction.
2Measurement precision
If geometric and semantic information processing is applied to estimate Manhattan layout, then the accuracy of 3D scene reconstruction is improved, but the processing complexity increases
Solution Approach 1:
The patent segments the complex task of 3D scene reconstruction into distinct processing stages: geometric information extraction, semantic information extraction, and layout estimation. By dividing the processing pipeline into separate modules that handle specific aspects independently, the system manages complexity more effectively while maintaining high accuracy through specialized processing at each stage.
Solution Approach 2:
The patent performs preliminary extraction and organization of geometric and semantic information from 2D images before conducting the actual layout estimation. By preparing and structuring the input data in advance, the system reduces the complexity of the subsequent layout estimation process while improving accuracy through more refined and organized input data.
Data Source
AI summary
A plurality of two-dimensional (2D) images of the scene is received. Geometric information and semantic information of each of the plurality of 2D images is determined. The geometric information indicates a detected line and a reference direction in the respective 2D image. The semantic information includes classification information of pixels in the respective 2D image. A layout estimation associated with the respective 2D image of the scene is determined based on the geometric information and the semantic information of the respective 2D image. A combined layout estimation associated with the scene is determined based on a plurality of the determined layout estimations associated with the plurality of 2D images of the scene. The Manhattan layout associated with the scene is generated based on the combined layout estimation. The Manhattan layout includes at least a three-dimensional (3D) shape of the scene that includes wall faces orthogonal with respect to each other.