Edge-Assisted AR Image Offloading for Fresh Tracking Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality devices face substantial processing power requirements for tasks like object detection, leading to delayed and stale image processing results due to uplink and downlink transfer times, causing tracking errors and unrealistic AR imagery.
Innovation Solution
An offloading schedule is determined for AR devices to strategically select which image frames to offload to a computing device, minimizing tracking stride and staleness of processing results by adapting between sending older frames to keep the computing device busy and stalling the pipeline for more recent frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If image frames are continuously offloaded to the computing device for processing, then the processing power requirement is met, but the tracking stride increases and results become stale due to uplink and downlink transfer times
Solution Approach 1:
The system implements periodic offloading by selecting specific image frames at predetermined intervals or based on timing criteria, rather than continuously offloading all frames. This periodic approach allows the AR device to maintain local processing for intermediate frames, reducing the tracking stride and keeping results fresher while still utilizing cloud processing power periodically.
Solution Approach 2:
The offloading schedule is dynamically adjusted based on current tracking stride measurements and system state. The system monitors the relationship between frame timing, processing delays, and tracking accuracy, then adapts the offloading decisions in real-time to optimize both processing power utilization and minimize time loss.
2Productivity
If older image frames are offloaded to keep the computing device busy, then processing throughput is improved, but the staleness of processing results increases causing tracking errors
Solution Approach 1:
The system changes the temporal parameter of frame selection by introducing variable offloading intervals and skipping certain frames based on timing criteria. This parameter adjustment allows the system to balance between keeping the computing device continuously busy (improving throughput) and selecting frames that minimize tracking stride (maintaining reliability).
Solution Approach 2:
The system performs preliminary evaluation of frame timing and tracking state before making offloading decisions. By anticipating the impact of offloading certain frames on tracking stride, the system can pre-determine optimal offloading schedules that maintain both throughput and tracking accuracy.
3Reliability
If the AR device processes all image frames locally, then tracking accuracy is maintained, but substantial processing power is required causing device complexity and energy consumption
Solution Approach 1:
The system extracts the heavy processing burden by offloading specific image frames to a remote computing device, retaining only essential local processing functions. This extraction allows the AR device to maintain tracking accuracy for critical frames while reducing device complexity and energy consumption by leveraging external processing resources.
Solution Approach 2:
The system implements a hybrid processing architecture that combines local AR device processing with remote cloud computing capabilities. This multi-functional approach allows the system to adaptively choose between local and remote processing based on frame importance, timing, and resource availability, reducing the processing power requirement of the AR device itself.
Data Source
AI summary
An offloading schedule for offloading image frames that are to be generated by an AR device in a subsequent time period is determined, the offloading schedule identifying certain image frames generated in the subsequent time period that are to be offloaded and certain image frames generated in the subsequent time period that are not to be offloaded, the offloading schedule being selected from a plurality of offloading schedules based on a tracking stride of the offloading schedule. The AR device sends to a computing device at least some of the image frames generated in the subsequent time period in accordance with the offloading schedule.