AR System Using Pre-computed Point Clouds for Real-time Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality systems face challenges in balancing the correlation between real and virtual objects in real-time while maintaining acceptable processing burdens, particularly when dressing 3D structures with virtual content, which can degrade the ability to reproduce and maintain virtual objects in real-time due to the computational power required.
Innovation Solution
The system generates and applies virtual content by processing visual information from real-world objects or environments, using initial rough location estimates to identify 3D point cloud models, texture data, and façade data, and tracks changes in device localization to accurately display virtual content alongside real-time captured images, employing techniques like Structure from Motion (SfM) and simultaneous location and mapping (SLAM) with simplified point cloud models and inertial measurement units (IMUs for enhanced tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If complex image capture information is integrated and matched with augmented reality information to achieve seamless presentation, then the quality and realism of AR display is improved, but the computational power required and processing burden increases
Solution Approach 1:
The system performs preliminary actions by capturing images and generating 3D point cloud models and texture maps in advance before the AR display is needed. These pre-processed models are stored and can be quickly retrieved and applied during AR operations, reducing the computational burden in real-time while maintaining high display quality.
Solution Approach 2:
The system creates simplified 3D point cloud models and texture maps as copies of the real-world environment. These digital copies serve as lightweight representations that can be manipulated and rendered efficiently without requiring complex real-time processing of actual image capture data, thus reducing computational power requirements while preserving visual fidelity.
2Manufacturing precision
If 3D structures are dressed with virtual content to enhance realism, then the immersion and quality of AR experience is improved, but the ability to reproduce and maintain virtual objects in real-time degrades due to computational requirements
Solution Approach 1:
The system generates 3D point cloud models, texture maps, and façade data in advance as preliminary preparations. These pre-computed models are stored in databases and can be rapidly retrieved and applied during AR operations, enabling high-quality virtual content dressing without compromising real-time reproduction capability.
Solution Approach 2:
The system creates simplified digital copies of real-world structures using 3D point cloud models and texture maps. These copies are computationally efficient representations that can be dressed with virtual content and rendered in real-time without requiring complex processing, thus maintaining both quality and productivity.
3Measurement precision
If accurate localization tracking is implemented to maintain correlation between real and virtual objects, then the precision and stability of AR display is improved, but the processing burden and system complexity increases
Solution Approach 1:
The system introduces 3D point cloud models and pre-computed environment data as intermediary elements between the physical environment and the AR display. These intermediaries provide stable reference frames for localization tracking, enabling accurate correlation between real and virtual objects without requiring complex real-time tracking algorithms, thus improving precision while managing system complexity.
Data Source
AI summary
The disclosure concerns an augmented reality method in which visual information concerning a real-world object, structure or environment is gathered and a deformation operation is performed on that visual information to generate virtual content that may be displayed in place of, or additionally to, real-time captured image content of the real-world object, structure or environment. Some particular embodiments concern the sharing of visual environment data and/or information characterizing the deformation operation between client devices.