Foreground Occluder Detection via Location-Aware Color Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality systems face issues with occlusions caused by users interacting with virtual objects, as virtual objects do not naturally become obscured by occluders like real objects, breaking the illusion and providing an unsatisfactory user experience.
Innovation Solution
A system that detects foreground occluders in an image stream, uses location information to determine expected features, partitions pixels into foreground and background regions, and trains a color model to classify pixels, ensuring virtual objects are rendered appropriately behind occluders.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual objects are rendered into the image stream, then the augmented reality content is enriched, but the illusion is broken when occluders block real objects but not virtual objects
Solution Approach 1:
The patent introduces an intermediary occlusion detection mechanism that analyzes the image stream to identify foreground occluders. This intermediary system determines which virtual objects should be obscured by occluders, mediating between the rendered virtual objects and the real-world occlusion events to maintain illusion consistency.
Solution Approach 2:
The system performs preliminary occlusion detection and analysis before final rendering decisions. By identifying occluders and determining their impact on virtual objects in advance, the system can pre-adjust the visibility and positioning of virtual objects to maintain the illusion that they are part of the real scene.
2Ease of operation
If the system detects and processes occluders to maintain illusion, then the user experience is improved, but the computational complexity increases
Solution Approach 1:
The patent segments the image stream into foreground and background regions, and further divides virtual objects into occluded and non-occluded categories. This segmentation allows the system to process only relevant portions of the image and virtual objects, reducing overall computational complexity while maintaining user experience.
Solution Approach 2:
The system applies partial processing by focusing computational resources only on regions where occluders are detected and on virtual objects that may be affected. Rather than processing the entire image stream and all virtual objects uniformly, the system performs selective processing only where needed to maintain the illusion.
Data Source
AI summary
A system for providing augmented reality detects foreground occluders in an image of a video stream. One or more virtual objects are then rendered appropriately with respect to the occluders. Location information associated with the image is used to retrieve a three dimensional representation of the location where the image was taken. Features that are expected to appear in the image based on the three dimensional location but that cannot be located are used to determine regions of the image that are likely to include foreground occluders. Pixels in these regions are used to train a color model that classifies pixels as either part of the background of the image or part of one or more foreground occluders. The pixels in the image are classified using the model, and one or more virtual objects are rendered so that they appear behind any foreground occluders.