Display Device Characterization for Augmented Reality Compositing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality applications face challenges in compositing virtual objects with real scenes that include display devices, as existing methods fail to accurately account for display device characteristics like specular reflections and color reproduction, leading to unrealistic virtual object integration.
Innovation Solution
A method where both captured and displayed images are processed together to determine characteristics of the display device, such as specular reflections and color reproduction functions, allowing for improved realism by accounting for these factors in the composition of virtual objects within the real scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If virtual objects are composited in captured videos without considering display device characteristics, then the processing complexity is reduced, but the realism of virtual object integration deteriorates
Solution Approach 1:
The patent performs preliminary characterization of the display device by capturing reference images of the display showing known test patterns. These characterization results (including color reproduction functions and specular reflection properties) are stored and reused for subsequent augmented reality compositing operations, avoiding repeated complex measurements while maintaining high realism
Solution Approach 2:
The patent introduces an intermediary processing step that separates the display device characterization from the virtual object compositing. A characterization module first extracts display properties (color reproduction, specular reflections) from captured images, then these properties are used as parameters in the compositing module to achieve realistic integration without directly coupling the two complex operations
2Measurement precision
If display device characteristics are determined through image comparison methods, then the measurement precision of display characteristics is improved, but the difficulty of detecting and measuring increases
Solution Approach 1:
The patent uses a camera to capture images of the display device, creating a digital copy of the visual output. By comparing the captured image with the original source image or reference patterns, the system extracts display characteristics (color reproduction accuracy, specular reflections) without requiring direct physical measurement instruments, thus improving measurement precision while keeping detection relatively simple
Solution Approach 2:
The patent implements a feedback mechanism where captured images of the display are analyzed to determine display characteristics, and this information is fed back to adjust the compositing parameters. The system iteratively refines the characterization by comparing expected vs. actual display output, improving measurement precision through feedback-driven optimization
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
In situation where a capturing device is capturing a first image of a scene comprising a display device displaying a second image, a salient idea is to send both the captured first image and the second image, being displayed by the display device to a processing device, for comparatively processing the first and the second image to determine a characteristic associated with the display device. Being able to process the first image (comprising a displayed second image) together with the original second image advantageously allows to determine various characteristics associated with the display device that altered the displayed second image within the first image, making it different from the initial second image. Characteristics associated with the display device are for example a specular reflection occurring on the display device in the scene and disturbing the capture of the scene. In a second and non-limiting example a characteristic of the display device is the color reproduction function of the display device that makes the displayed second image within a displayed first image be different from the displayed second image in the scene.