AR Plane Border Detection Using Occlusion and Support Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining the true border of a surface in augmented reality environments, such as a tabletop, fail to distinguish between the physical border of an object and the border of the reconstruction, which is crucial for accurate interaction and occlusion in AR applications.
Innovation Solution
A system and method that constructs support and occlusion maps using depth images from RGBD cameras to identify the true border of a planar object by partitioning the surface into cells, counting points, and classifying segments based on occlusion and support gradients, allowing for the determination of the physical border within an AR environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current methods are used to extract points corresponding to a single object, then a border can be found, but the method cannot distinguish between the physical border of the object and the border of the reconstruction
Solution Approach 1:
The patent segments the problem into two distinct analysis components: support map analysis (identifying where the plane exists) and occlusion map analysis (identifying where the plane is blocked). By processing these two maps separately and combining their results, the system can distinguish between true physical borders and artificial reconstruction borders, resolving the information loss in border detection.
Solution Approach 2:
The patent introduces intermediary data structures (support map and occlusion map) that mediate between the raw depth image data and the final border determination. These intermediary maps encode different aspects of surface information, allowing the system to reason about physical versus reconstruction borders through logical combination rather than direct analysis.
2Area of stationary object
If multiple depth images are captured and combined to create an extensive 3D model, then a sampled representation of the scene surface is obtained, but the method cannot determine if a border is the true border of the surface or only the border of the recorded data
Solution Approach 1:
The patent divides the large-scale 3D reconstruction problem into localized analysis units by processing support and occlusion information for each potential border segment independently. This segmentation allows the system to maintain accurate border authentication even when analyzing extensive scene areas, as each local decision can be made with full contextual information.
Solution Approach 2:
The patent transitions from analyzing only the geometric extent of the reconstructed surface to analyzing the visibility relationships in a new dimension. By constructing occlusion maps that encode which surfaces block views of other surfaces, the system gains an additional analytical dimension for determining border authenticity that is independent of the physical scene coverage area.
3Adaptability or versatility
If games need to know that a surface can grow at the border of the reconstruction, but not at the physical borders of the surface, then accurate interaction is required, but current methods cannot provide this distinction
Solution Approach 1:
The patent segments borders into classified types (physical versus reconstruction borders) based on support and occlusion map analysis. This segmentation enables different interaction behaviors for different border types, allowing surfaces to grow at reconstruction borders while maintaining fixed boundaries at physical borders, thus achieving the required adaptability with precise classification.
Solution Approach 2:
The patent applies different qualities/properties to different parts of the reconstructed surface based on border classification. Physical borders receive the property of being fixed and non-growable, while reconstruction borders receive the property of being extendable. This local differentiation of border properties enables the system to provide accurate interaction behavior tailored to each border type.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Methods, systems, computer-readable media, and apparatuses for constructing a representation of a planar object are presented. In some embodiments, techniques for constructing a representation of a planar object are disclosed. According to some embodiments, a method for constructing a representation of a planar object may include obtaining a depth image of a physical scene as captured from a viewing position. The depth image may comprise a plurality of depth values and corresponding to a plurality of points in the physical scene. The method may further include identifying a planar surface along which the planar object is estimated to be positioned. Furthermore, the method may include constructing a support map. Moreover, the method may include constructing an occlusion map, the occlusion map indicating portions of the planar surface where the planar object is missing. Subsequently, the method may include constructing a representation of at least one boundary of the planar object, using the occlusion map.