Deformable Surface Marker Tracking in Augmented Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality techniques fail to effectively track three-dimensional coordinates and orientations of two-dimensional planar markers on deformable surfaces, such as soft objects or moving water, as they do not account for the surface's distortions and changes in texture and shape during motion.
Innovation Solution
The method involves generating polygon meshes to track the marker's three-dimensional positions, orientations, and deformations by identifying and matching key-points across video frames, using a key-point matching score filter, adaptive threshold, and motion detection to enhance stability and accuracy, and restricting key-point searches within a mesh block to reduce noise and lighting fluctuations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional marker tracking techniques are used on deformable surfaces, then the implementation is simple, but the tracking accuracy and stability deteriorate due to surface distortions and texture changes
Solution Approach 1:
The patent divides the marker into multiple key-point locations and represents the deformable surface as a mesh of polygons. Each key-point is tracked independently, and the mesh segments the surface into trackable units that can deform independently, allowing accurate tracking despite surface distortions.
Solution Approach 2:
The patent transitions from traditional two-dimensional planar marker tracking to three-dimensional deformable surface tracking by introducing depth information and spatial coordinates. The mesh structure adds a dimensional layer that captures the three-dimensional deformation of the surface, enabling accurate tracking in 3D space rather than just 2D plane.
2Reliability
If key-point matching is performed across all video frames without restrictions, then the coverage is comprehensive, but the noise and lighting fluctuations increase matching errors
Solution Approach 1:
The patent applies local quality by restricting key-point searches to specific mesh blocks surrounding previously identified key-points, rather than searching the entire image. This localizes the search area to regions where key-points are expected to appear, reducing the impact of noise and lighting fluctuations from other parts of the image while maintaining matching reliability.
3Measurement precision
If the marker texture and shape changes are not accounted for, then the processing is simpler, but the virtual object placement accuracy deteriorates on deformable surfaces
Solution Approach 1:
The patent implements dynamics by creating a deformable mesh model that can change its shape and texture coordinates according to the actual deformation of the marker on the deformable surface. The mesh vertices are adjusted based on key-point matching results, allowing the marker representation to dynamically adapt to surface deformations, which ensures accurate virtual object placement even when the marker's appearance changes.
Data Source
AI summary
A computer implemented method for tracking a marker on a deformable surface in augmented reality (AR) applications, comprising: detecting image-key-points in a currently processed video frame of a video-captured scene; performing key-point-correspondence searching and matching the image-key-points with model-key-points are identified from an original image of the marker, comprising: calculating an key-point matching score for each image-key-point; applying a key-point matching score filter on the key-point matching scores; restricting the searching of the image-key-points in the currently processed video frame to within same mesh block determined in a previously processed video frame of the captured video frames; and applying adaptive thresholds on the key-point matching scores in determining successful matches of the image-key-points; performing motion detection of the marker in the video-captured scene and halting the application of the key-point matching score filter and suspending the restriction on the image-key-point searching if the marker is in significant movement.