AR Occlusion via 3D Spatial Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems struggle to represent virtual objects in a real environment in a way that accurately conceals real objects, leading to unrealistic collisions and the inability to remove or reposition spurious real objects without physical intervention.
Innovation Solution
A method combining 3D reconstruction data with 2D segmentation techniques to identify and remove spurious objects from the image, allowing for realistic occlusion and repositioning of virtual objects within the real environment, using a recording device and data processing system to generate and merge images while considering segmentation data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If virtual objects are always represented in the foreground for perspectively correct representation, then the virtual object can be clearly seen, but real objects cannot be concealed and the representation becomes unrealistic
Solution Approach 1:
The patent transitions from 2D image processing to 3D spatial reasoning by introducing depth information and occlusion zones. Virtual objects are placed in three-dimensional space relative to real objects, allowing the system to determine when a virtual object should be concealed behind a real object based on their spatial relationships, thus achieving both perspective correctness and visual realism simultaneously
2Adaptability or versatility
If real objects are removed from the image to allow virtual object placement, then product suitability can be assessed, but the system complexity increases and requires additional processing
Solution Approach 1:
The patent extracts only the necessary information (occlusion zones and spatial relationships) from the real environment rather than removing or heavily processing the entire scene. By identifying and storing minimal geometric data about real objects that could potentially occlude virtual objects, the system enables versatile product placement assessment while keeping processing complexity manageable
3Extent of automation
If virtual objects are merged with the real environment image, then an augmented reality effect is achieved, but collisions with real objects cannot be realistically represented
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously checks the spatial relationship between virtual and real objects, determining occlusion status based on their positions and dimensions. This feedback loop allows the augmented reality system to dynamically adjust whether virtual objects should be visible or concealed, ensuring realistic collision representation while maintaining the automated augmented reality effect
Data Source
AI summary
The invention relates to a method for representing a virtual object in a real environment, having the following steps: generating a two-dimensional image of a real environment by means of a recording device, ascertaining a position of the recording device relative to at least one component of the real environment, segmenting at least one area of the real environment unmarked in reality in the two-dimensional image for identifying at least one segment of the real environment in distinction to a remaining part of the real environment while supplying corresponding segmentation data, and merging the virtual object with the two-dimensional image of the real environment with consideration of the segmentation data such that at least one part of the segment of the real environment is removed from the image of the real environment. The invention permits any collisions of virtual objects with real objects that occur upon merging with a real environment to be represented in a way largely close to reality.