AR Marker Tracking with Optical Flow for Faster Scene Overlay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) technologies consume significant device resources and time in analyzing and recognizing complex video frames, particularly in complicated real environments, leading to inefficient processing and recognition of virtual content.
Innovation Solution
The method employs marker images with high resolution and regularity, utilizing optical flow tracking and homography matrices to track feature points across video frames, supplemented by template matching when necessary, to efficiently superimpose virtual content on real objects, reducing resource consumption and recognition time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire video frame is analyzed and recognized using image recognition technology, then the position of image objects can be determined accurately, but the processing time and device resources consume significantly increase
Solution Approach 1:
The patent divides the video frame into multiple regions of interest (ROIs) based on pre-detected marker positions. Instead of analyzing the entire frame, the system segments the frame into smaller regions around markers, extracting features only from these relevant areas. This segmentation dramatically reduces processing time while maintaining accurate marker position recognition, directly resolving the contradiction between recognition accuracy and processing time.
Solution Approach 2:
The patent performs preliminary actions by detecting markers in previous frames and storing their position information before processing current frames. This preliminary marker detection establishes a reference framework that guides subsequent feature extraction and tracking in current frames, eliminating the need to analyze the entire current frame from scratch and reducing computational time while maintaining precision.
2Adaptability or versatility
If the entire video frame is analyzed and recognized, then comprehensive object detection is achieved, but device software and hardware resources are greatly consumed
Solution Approach 1:
The patent applies local quality by focusing computational resources on specific local regions around detected markers rather than uniformly processing the entire frame. The system extracts and analyzes features only from regions containing markers or adjacent to them, while skipping feature extraction in empty regions. This localized approach maintains comprehensive object detection capability for marked objects while significantly reducing overall device resource consumption.
Solution Approach 2:
The patent extracts only the necessary information (marker positions and surrounding features) from the video frame, discarding unnecessary data from regions without markers. By taking out and processing only the essential parts of the frame containing markers, the system achieves comprehensive detection of marked objects while reducing software and hardware resource consumption compared to full-frame analysis.
3Manufacturing precision
If image recognition technology is used to recognize objects in each video frame, then virtual content can be superimposed at correct positions, but the processing efficiency decreases significantly in complicated real environments
Solution Approach 1:
The patent introduces marker images as intermediary elements between the real world and virtual content. These markers serve as mediators that provide stable, easily detectable reference points in the real environment. By using markers as intermediaries, the system achieves accurate virtual content superposition without needing to perform complex image recognition on the entire frame, thereby maintaining high processing efficiency even in complicated real environments.
Solution Approach 2:
The patent uses simplified marker images as copies or representations of real-world objects or surfaces. Instead of performing complex recognition on the actual complex environment, the system detects these simplified marker copies and uses their known positions to guide virtual content superposition. This copying approach maintains superposition accuracy while dramatically improving processing efficiency by replacing complex recognition tasks with simple marker detection.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
An augmented reality scene processing method, comprising: obtaining first feature point information, related to a target mark image, in a first video frame, the first video frame being captured by a camera device; tracking first feature points corresponding to the first feature point information according to an optical flow tracking algorithm, and determining second feature point information in a second video frame captured by the camera device; constructing an injection matrix of the second video frame and the target mark image according to the second feature point information and source feature points of the target mark image; and performing augmented reality processing on the second video frame according to the injection matrix.