3D Point Cloud Rendering Using Low-Resolution Mesh
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for rendering 3D point clouds of physical environments in extended reality environments face challenges in providing real-time, stable, and efficient views, often resulting in high processing costs and occlusion/instability issues.
Innovation Solution
The use of a low-resolution 3D mesh to select and render a subset of 3D point cloud points, ensuring occluded points are not used and employing depth information for inpainting, while also generating textured planar surfaces to enhance the view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a high-resolution 3D mesh is used for rendering, then rendering quality and detail are improved, but processing time and computational cost increase significantly
Solution Approach 1:
The patent segments the rendering process into two distinct stages: (1) generating a low-resolution 3D mesh at high frame rate for real-time rendering, and (2) generating a high-resolution 3D point cloud at lower frame rate for quality enhancement. This segmentation allows the system to achieve both real-time performance and high rendering quality without the computational burden of processing high-resolution data at full speed.
Solution Approach 2:
The patent applies partial action by using a low-resolution mesh for the majority of rendering operations where high frame rate is critical, and only applying high-resolution processing to specific regions or frames where quality is most important. This selective approach maintains real-time performance while incorporating high-quality rendering where needed.
2Quantity of substance
If all 3D point cloud points are used for rendering, then completeness of the environment representation is improved, but occlusion issues and instability increase
Solution Approach 1:
The patent introduces a low-resolution 3D mesh as an intermediary structure that mediates between the high-resolution point cloud and the final rendered image. The mesh serves as a coarse guide to determine which point cloud points are visible and should be rendered, effectively filtering out occluded points and stabilizing the rendering process while maintaining environmental completeness.
Solution Approach 2:
The patent applies local quality by differentiating the treatment of different point cloud regions. Points are selectively rendered based on their spatial relationship with the low-resolution mesh, with higher detail applied to visible regions and reduced detail or exclusion applied to occluded regions. This localized approach maintains reliability by avoiding occlusion issues while preserving completeness where visible.
3Speed
If real-time rendering is implemented, then frame rate and responsiveness are improved, but processing cost and computational load increase
Solution Approach 1:
The patent segments computational work by performing heavy processing (high-resolution mesh generation) offline or at lower frame rates, while real-time rendering operates at lower resolution. This division allows the system to maintain high frame rates during interactive rendering while still producing high-quality output, effectively managing computational power requirements.
Solution Approach 2:
The patent dynamically changes rendering parameters based on requirements, switching between low-resolution high-frame-rate mode for real-time interaction and high-resolution mode for quality enhancement. This parameter adaptation allows the system to optimize the balance between frame rate and computational cost based on the specific rendering context.
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that generate 2D views of a 3D environment using a 3D point cloud where the cloud points selected for each view are based on a low-resolution 3D mesh. In some implementations, a 3D point cloud of a physical environment is obtained, the 3D point cloud including points each having a 3D location and representing an appearance of a portion of the physical environment. Then, a 3D mesh is obtained corresponding to the 3D point cloud, and a 2D view of the 3D point cloud from a viewpoint is generated using a subset of the points of the 3D point cloud, where the subset of points is selected based on the 3D mesh.