AR Depth Image Blending for Stable 3D Object Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) systems suffer from inaccurate and unstable depth data, leading to inconsistent rendering of 3D objects relative to real-world objects, resulting in a less than desirable user experience.
Innovation Solution
Storing depth data associated with real-world objects in a buffer and using this data to accurately render 3D objects by blending and replacing missing depth information with valid data, utilizing surface elements (surfels) to enhance depth and position processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current AR systems use depth data from depth cameras, then real-time 3D reconstruction is achieved, but depth data becomes inaccurate and unstable causing incorrect rendering of 3D objects
Solution Approach 1:
The system performs preliminary actions by capturing and storing depth images and surfels before the actual rendering occurs. Historical depth data is accumulated in advance to create a comprehensive depth buffer that can be referenced during rendering, ensuring accurate depth information is available even when current depth camera data is unreliable.
Solution Approach 2:
The patent introduces an intermediary approach by using surfels (surface elements) as a mediator between the depth camera and the rendering system. Surfels serve as intermediate representations that bridge the gap between raw depth data and final rendering, allowing for more accurate and stable depth information to be derived and used in the rendering process.
2Speed
If depth data is processed in real-time, then rendering responsiveness is improved, but depth information becomes unstable and inaccurate
Solution Approach 1:
The system performs preliminary actions by capturing and storing depth images and surfels before the actual rendering occurs. Historical depth data is accumulated in advance to create a comprehensive depth buffer that can be referenced during rendering, ensuring accurate depth information is available even when current depth camera data is unreliable.
Solution Approach 2:
The patent implements feedback mechanisms where the system continuously monitors and refines depth information based on historical data and rendering results. The depth buffer is updated and refined through feedback loops that compare current depth data with historical surfel data, adjusting and correcting inaccuracies in real-time to improve both responsiveness and stability.
3Measurement precision
If multiple depth images are stored and processed, then depth accuracy is improved, but system complexity and data processing requirements increase
Solution Approach 1:
The patent applies segmentation by dividing the depth data into discrete surfels (surface elements) that represent specific portions of the real-world scene. This segmentation allows the system to process and store depth information in manageable units, reducing the complexity of handling large volumes of depth data while maintaining high accuracy through the accumulated surfel representations.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
A method including receiving a first depth image associated with a first frame at a first time of an augmented reality (AR) application, the first depth image representing at least a first portion of a real-world space storing the first depth image receiving a second depth image associated with a second frame at a second time, after the first time, of the AR application, the second depth image representing at least a second portion of the real-world space generating a real-world image by blending, at least, the stored first depth image with the second depth image receiving a rendered AR object combining the AR object in the real-world image and displaying the real-world image combined with the AR object.