AR Reprojection Engine Using Per-Pixel Depth and Motion Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems face challenges in accurately projecting and reprojecting virtual content onto a display that also shows the real-world environment, leading to reprojection errors due to changes in user headpose and motion, which can result in occlusions and inaccuracies in depth perception.
Innovation Solution
The implementation of a reprojection engine that utilizes per-pixel depth information, changes in headpose, three-dimensional motion vectors, and surface color data to reproject virtual content with reduced errors and improved accuracy between frames, while also modulating resource consumption on-demand.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional projection methods are used in augmented reality systems, then the system complexity is reduced, but reprojection accuracy deteriorates due to headpose changes and motion
Solution Approach 1:
The patent segments the projection process into multiple components: depth buffer generation, reprojection calculation, headpose tracking, and frame rendering. By dividing the complex reprojection task into manageable segments, the system achieves higher accuracy through per-pixel depth processing while maintaining organizational complexity at a manageable level.
Solution Approach 2:
The system performs preliminary actions by pre-calculating depth buffers and motion vectors before the actual reprojection occurs. Depth information is extracted and stored in advance, allowing the reprojection engine to quickly apply transformations when headpose changes, thereby improving accuracy without adding real-time computational complexity.
2Measurement precision
If per-pixel depth processing is implemented to improve reprojection accuracy, then measurement precision improves, but computational resource consumption increases
Solution Approach 1:
The patent applies partial action by processing only the necessary pixels for reprojection rather than all pixels in the frame. The reprojection engine selectively updates regions where virtual content overlaps with real-world features, reducing computational load while maintaining depth perception accuracy in critical areas.
Solution Approach 2:
The system uses self-service by leveraging existing depth buffer data and motion vector information that are already generated during normal rendering operations. These pre-computed data structures serve the dual purpose of standard rendering and reprojection calculations, eliminating redundant computations and reducing energy consumption.
3Reliability
If reprojection calculations are performed frequently to reduce errors, then reliability improves, but productivity deteriorates due to increased processing time
Solution Approach 1:
The patent implements periodic action by performing full reprojection calculations at specified intervals rather than continuously. The reprojection engine updates virtual content positioning periodically based on accumulated headpose changes, maintaining projection coherence over time while allowing frame rendering to proceed at normal speeds between update intervals.
Data Source
AI summary
Methods and systems are provided for a reprojection engine for augmented-reality devices. The augmented-reality device projects virtual content within a real-world environment. The augmented-reality device tracks a six degrees of freedom headpose of the augmented-reality device, depth information of the virtual content, motion vectors that correspond to movement of the virtual content, and a color buffer for a reprojection engine. The reprojection engine generates a reprojection of the virtual content defined by an extrapolation of a first frame using the headpose, the depth information, motion vectors, and the color surface data structure. The reprojected virtual content continues to appear as if positioned with the real-world environment regardless of changes in the headpose of the augmented-reality device or motion of the virtual content.