Augmented Reality Fiduciary Marker Segmentation for Occlusion Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality systems face challenges in maintaining accurate tracking and interaction with virtual objects when fiduciary markers are occluded by user gestures, leading to unnatural movement and frustration, especially in adverse lighting conditions or low-resolution camera scenarios.
Innovation Solution
The use of large fiduciary markers covering a significant portion of each page, spatial redundancy, clear border markings, and color schemes that differentiate between the book and user's skin tone, combined with gesture recognition methods like cupping hands to pick up virtual objects, ensures robust interaction and tracking even when markers are partially occluded.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a fiduciary marker is placed on the surface for AR tracking, then the system can accurately determine scale and orientation, but the marker may be occluded by user gestures causing loss of tracking accuracy
Solution Approach 1:
The fiduciary marker is divided into multiple distinct features or regions (e.g., corners, edges, internal patterns) that can be independently detected. This segmentation allows the system to maintain tracking accuracy even when parts of the marker are occluded, as long as sufficient features remain visible.
Solution Approach 2:
The system performs preliminary detection and tracking of the fiduciary marker before user interaction occurs. By establishing the marker's position, scale, and orientation in advance, the system can maintain accurate AR augmentation even during brief occlusions that occur during user gestures.
2Measurement precision
If the video camera has high resolution and good lighting, then marker detection is more accurate, but the system performance and processing speed decrease
Solution Approach 1:
The marker design incorporates distinct segmented features that can be rapidly detected using simplified algorithms. The high-contrast borders and asymmetric patterns are broken down into discrete detectable elements that reduce computational complexity while maintaining detection accuracy.
Solution Approach 2:
The fiduciary marker uses extreme parameter values - very high contrast borders and distinctive asymmetric patterns - that make it easily distinguishable from the background and other objects. This allows for fast detection with lower computational requirements, enabling real-time processing even with detailed marker features.
3Reliability
If the fiduciary marker has high contrast and asymmetric patterning, then the marker is more robust to lighting conditions, but the marker design complexity increases
Solution Approach 1:
The fiduciary marker incorporates asymmetric patterning that provides unique orientation information. The asymmetric design allows the system to easily determine the marker's rotational orientation without complex algorithms, as the asymmetric pattern naturally indicates the correct orientation through its unique configuration.
Solution Approach 2:
The marker uses high-contrast color schemes with distinct borders and internal patterns that maintain visibility across varying lighting conditions. The strong color differentiation and contrast ensure the marker remains detectable and distinguishable from the background and foreground objects regardless of lighting changes.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A method of augmented reality interaction for repositioning a virtual object on an image of a surface comprises capturing successive video images of the surface and first and second control objects, defining an interaction start area over the surface with respect to the virtual object, detecting the first and second control objects in successive video images, detecting whether the first and second control objects are brought together over the interaction start area, and if so, analysing a region of successive video images using optical flow analysis to determine the overall direction of motion of the first and second control objects and augmenting the video image to show the virtual object being held by the first and second control objects, in which augmenting the video image itself comprises superposing a graphical effect on the video image prior to superposition of the virtual object, such that the graphical effect visually disconnects the virtual object from the video image in the resulting augmented image.