Confidence-Guided AR Image Blending for Accurate Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in generating realistic images due to imprecise or incomplete depth information from capture devices, leading to erroneous depth comparisons and visual artefacts in real-time applications.
Innovation Solution
An augmented reality processing system that identifies confidence and uncertainty regions based on color similarity and depth values, using blending factors to combine images, thereby reducing errors in occlusion rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If depth information is obtained from a single point using a capture device, then processing time is reduced for real-time applications, but depth measurement precision deteriorates leading to erroneous depth comparisons
Solution Approach 1:
The image is divided into confidence regions and uncertainty regions based on depth measurement reliability. Confidence regions use single-point depth measurements for fast processing, while uncertainty regions are identified and handled separately using multi-point scanning data to correct depth errors, thus resolving the contradiction between speed and precision.
Solution Approach 2:
A depth confidence map is introduced as an intermediary data structure that stores reliability information for each pixel's depth measurement. This map guides the rendering process to use corrected depth values from multi-point scanning only where needed (in uncertainty regions), maintaining real-time performance while improving precision where necessary.
2Measurement precision
If multi-point scanning is performed to generate accurate depth maps, then depth measurement precision is improved, but processing time increases making it unsuitable for real-time applications
Solution Approach 1:
Instead of performing multi-point scanning for the entire scene, the system applies depth correction only to uncertainty regions where single-point measurements are unreliable. This partial application of the time-consuming correction process maintains accuracy where needed while preserving real-time performance overall.
Solution Approach 2:
Depth confidence information from multi-point scanning is pre-calculated and stored in a confidence map before real-time rendering. During real-time processing, the system only needs to query this pre-computed confidence information and apply corrections to specific uncertainty regions, rather than performing full multi-point scanning for each frame.
3Manufacturing precision
If depth values are compared to determine occlusion, then rendering accuracy is improved, but visual artefacts increase when depth measurements are imprecise
Solution Approach 1:
The rendering quality is made non-uniform across the image: confidence regions use standard depth comparison for efficient rendering, while uncertainty regions use corrected depth values from multi-point scanning to eliminate artefacts. This local differentiation of quality ensures high accuracy where needed without incurring the full processing cost everywhere.
Data Source
AI summary
A method for generating an augmented reality image from first and second images, wherein at least a portion of at least one of the first and the second image is captured from a real scene, identifies a confidence region in which a confident determination as to which of the first and second image to render in that region of the augmented reality image can be made, and identifies an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the augmented reality image. At least one blending factor value in the uncertainty region is determined based upon a similarity between a first colour value in the uncertainty region and a second colour value in the confidence region, and an augmented reality image is generated by combining, in the uncertainty region, the first and second images using the at least one blending factor value.