VST AR Extended-View Geometry Rendering for Low-Latency Wide-Angle Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
VST AR systems face challenges in generating and rendering images efficiently, leading to high processor intensity and noticeable latency, which affects user experience due to the difficulty in providing wider fields of view.
Innovation Solution
The method involves obtaining multiple see-through image frames using imaging sensors, generating a depth map with spatial and temporal consistency, projecting the 3D representation onto a curved surface, and mapping points to virtual view images for presentation on AR device displays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional image generation and rendering methods are used in VST AR systems, then processing can be performed with conventional approaches, but processor intensity becomes high and latency increases
Solution Approach 1:
The patent segments the image generation process into distinct modules: depth map generation from multiple image frames, 3D representation construction, curved surface projection, and virtual view synthesis. This segmentation allows each module to be optimized independently and processed in parallel where possible, reducing overall processing time and latency while maintaining rendering quality
Solution Approach 2:
The patent performs preliminary actions by pre-computing depth maps from captured image frames and pre-generating 3D representations of the environment before virtual view rendering is required. This preliminary processing of geometric data structures enables faster real-time rendering of virtual views, significantly reducing latency during actual AR display updates
2Adaptability or versatility
If conventional rendering approaches are used, then implementation is straightforward, but wider fields of view cannot be provided efficiently
Solution Approach 1:
The patent projects the 3D representation onto a curved surface rather than a flat plane, adding a dimensional transformation that enables wider field of view rendering. This curved surface projection technique allows the system to display extended-view geometries that encompass broader angular ranges while maintaining manageable rendering complexity through mathematical projection transformations
Solution Approach 2:
The patent creates virtual view images by mapping points from the projected 3D representation to generate multiple virtual camera views. This copying approach generates additional viewpoint information from a single 3D model, enabling wide field of view without requiring multiple physical sensors or complex multi-camera systems
Data Source
AI summary
A method includes obtaining multiple see-through image frames of an environment around an augmented reality (AR) device using multiple imaging sensors of the AR device. The method also includes generating a depth map based on the see-through image frames and generating a three-dimensional (3D) representation of the environment based on the depth map. The method further includes projecting the 3D representation onto a curved surface, mapping points of the projected 3D representation to multiple virtual view images, and presenting the virtual view images on one or more displays of the AR device. Generating the depth map may include generating an initial depth map using a trained machine learning model and modifying the initial depth map to provide both spatial consistency and temporal consistency in order to generate a refined depth map. The curved surface may include a portion of a cylindrical, spherical, or conical surface.