AR Object Detection Anchors for Stable Multi-Frame Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional object detection systems in augmented reality (AR) suffer from unstable bounding boxes that 'jump' during camera movement, leading to transient false positives, inability to detect non-unique objects within unique objects, and failure to track stable proposals in the world state.
Innovation Solution
A system that generates predictions for objects using object models, compares them with proposals using spatio-temporal tube intersection-over-union analysis, updates proposals based on overlap thresholds, filters for stability, and annotates images with stable object detections, while maintaining anchors in a datastore to track objects across multiple frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional object detection is used in AR, then object identification can be performed, but bounding boxes become unstable and jump during camera movement
Solution Approach 1:
The system performs preliminary actions by tracking object proposals across multiple video frames before final detection. It maintains a history of proposals and performs temporal filtering to establish stable object detections before rendering AR annotations, preventing bounding box jumps during camera movement.
Solution Approach 2:
The system ensures continuity by processing object detection across a continuous stream of video frames rather than independently analyzing each frame. It maintains proposal continuity through temporal tracking and only confirms detections that persist across multiple frames, ensuring stable and continuous object tracking.
2Reliability
If traditional object detection is used, then objects can be identified, but transient false positives occur due to instability
Solution Approach 1:
The system applies preliminary temporal filtering to proposals before final detection confirmation. It requires proposals to meet minimum frame duration thresholds and consistency criteria across multiple frames before being classified as valid detections, thereby eliminating transient false positives.
Solution Approach 2:
The system implements feedback mechanisms by continuously evaluating proposal stability across video frames. It uses temporal consistency metrics to feedback on detection validity, adjusting detection confidence based on whether proposals persist across multiple frames and maintain spatial consistency.
3Device complexity
If object detection is performed on each frame independently, then processing is simple, but inability to detect non-unique objects within unique objects occurs
Solution Approach 1:
The system performs preliminary temporal analysis of video frames to establish object hierarchies and relationships before final classification. By tracking proposals across multiple frames, it can differentiate non-unique objects within unique objects based on their temporal persistence and spatial relationships, enabling sophisticated object recognition without excessive complexity.
4Speed
If traditional detection without temporal filtering is used, then processing speed is fast, but stable proposals cannot be maintained in world state
Solution Approach 1:
The system performs preliminary temporal filtering and proposal validation before committing detections to world state. It pre-processes proposals through temporal consistency checks and frame-duration thresholds, ensuring that only stable, reliable detections are maintained in the world state, thereby balancing speed with reliability.
Data Source
AI summary
A system for object detection in an augmented reality scene includes a computing system adapted to receive a stream of images from a user device. For each image in a plurality of successive images from the stream of images, the computing system: generates predictions for objects identified in the image using object models from a datastore; compares the predictions with proposals from the datastore, the comparison including checking an overlap threshold of the predictions and the proposals; updates the proposals with the predictions if the overlap threshold is met, otherwise adds new proposals to the proposals using the predictions; filters the proposals to create stable object detections based on a stability threshold; annotates the image using the stable object detections; and sends the annotated image to the user device for presentation.