AR Depth Image Rendering via 3D World Model Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality devices face challenges in accurately displaying virtual objects in the real world due to inaccurate sensor data, requiring significant computational resources to determine object positions and structures, leading to unrealistic virtual object placement.
Innovation Solution
The use of a 3D world model generated from image data to estimate the physical pose of an AR device and render depth images, allowing for more accurate and efficient display of virtual objects by modifying captured images to appear as if the virtual objects are real, leveraging graphics processors for parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor data is used to determine object positions and structures in AR, then the AR device can display virtual objects in the real world, but the sensor data is inaccurate and requires significant computational resources to interpret
Solution Approach 1:
The patent introduces a pre-built 3D world model as an intermediary between the sensor data and the virtual object placement. Instead of directly interpreting raw sensor data to determine object positions, the system uses the 3D world model which has already been processed and structured to represent the physical environment. This intermediary provides accurate spatial relationships without requiring real-time computational interpretation of raw sensor data.
Solution Approach 2:
The 3D world model is constructed in advance through preliminary scanning and processing of the environment. By performing the complex computational work of creating an accurate spatial representation beforehand, the system avoids needing to perform this computation in real-time during AR rendering. The pre-processed model contains ready-to-use information about object positions, structures, and spatial relationships.
2Productivity
If real-time sensor data interpretation is performed to locate the AR device relative to environmental objects, then virtual objects can be positioned, but the process is computationally expensive and time-consuming
Solution Approach 1:
The system performs environment mapping and 3D model construction as a preliminary action before AR rendering begins. This upfront computational work creates a reusable spatial database that can be queried efficiently during runtime. Instead of continuously processing sensor data to understand the environment, the system once-off creates a 3D world model that enables rapid virtual object placement with minimal real-time computation.
Solution Approach 2:
The patent creates a digital copy of the physical environment in the form of a 3D world model. This virtual representation replicates the spatial structure, object positions, and geometric relationships of the real world. By working with this copied representation rather than continuously analyzing raw sensor data, the system achieves fast virtual object placement with reduced computational energy consumption.
3Reliability
If sensor data is used to determine environmental structures, then virtual objects can be integrated into the real world, but the displayed virtual objects often fail to appear realistic
Solution Approach 1:
The 3D world model serves as an intermediary that provides a more reliable and precise representation of the physical environment than raw sensor data alone. This structured model contains accurate geometric information, spatial relationships, and environmental features that enable realistic virtual object integration. The intermediary model bridges the gap between imperfect sensor measurements and the need for precise environmental understanding.
Solution Approach 2:
The patent replaces the mechanical interpretation process of analyzing raw sensor data with a more sophisticated approach using pre-processed 3D world models. Instead of relying on direct sensor-to-interpretation pipelines that lose precision, the system substitutes this with querying a structured 3D representation that preserves environmental accuracy. This substitution enables more reliable determination of where and how to place virtual objects for realistic integration.
Data Source
AI summary
An AR device displays virtual objects to users as part of an AR experience by generating a depth image by rendering the depth image from a three-dimensional (3D) world model. The AR device receives an image from a camera and estimates its physical pose in the real world when the image was captured. The AR device accesses the 3D world model and estimates a virtual pose within a 3D world model that corresponds to the estimated physical pose in the real world. The AR device uses the virtual pose to render the depth image using the 3D world model. The AR device may use a graphics processor to render the depth image from a camera view corresponding to the virtual pose. The AR device uses the depth image to present content to the user over the image captured by the camera.