Asynchronous Edge-Cloud Detection for Real-Time Video Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Edge computing devices face a tradeoff between low latency and reduced accuracy in object detection due to limited resources, while cloud computing provides high accuracy but high latency, making it difficult to achieve real-time, high-accuracy object detection in IoT devices like AR and MR headsets.
Innovation Solution
Implementing a hybrid approach where a lower-latency detection model operates at the edge and a higher-latency detection model operates at the cloud, with the edge model asynchronously updating its results using redundant cloud detections to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a larger and more accurate detection model is operated on cloud computing devices, then object detection accuracy is improved, but processing latency increases
Solution Approach 1:
The patent segments the object detection system into two parts: a lightweight detection model deployed at the edge device for real-time processing, and a larger, more accurate detection model deployed at the cloud computing device. The edge model handles immediate detection needs while the cloud model provides enhanced accuracy through asynchronous updates, effectively dividing the computational workload between local and remote systems.
Solution Approach 2:
The patent introduces an intermediary mechanism where the edge device maintains a cache of object detections and uses asynchronous updates from the cloud model to refine these detections. This intermediary approach allows the system to leverage the accuracy of the cloud model without suffering its latency penalties for real-time processing, as updates are applied asynchronously rather than synchronously.
2Loss of time
If a smaller detection model is operated on edge computing devices, then processing latency is reduced, but object detection accuracy decreases
Solution Approach 1:
The patent applies preliminary action by pre-processing video frames at the edge device using the lightweight detection model before transmitting them to the cloud. The edge device performs initial detection and tracking, maintaining a cache of object detections that can be asynchronously updated. This preliminary processing ensures real-time responsiveness while the cloud model refines accuracy in the background.
Solution Approach 2:
The patent implements feedback mechanisms where the cloud computing device sends asynchronous updates to the edge device based on refined detections from the larger model. These feedback updates continuously improve the accuracy of object detections at the edge without requiring the edge device to run the computationally intensive cloud model, creating a continuous improvement loop that maintains both low latency and high accuracy.
3Measurement precision
If redundant object detections are performed by both edge and cloud models, then object detection accuracy is improved, but computing resource usage increases
Solution Approach 1:
The patent extracts the redundant detection computations from the edge device by implementing asynchronous updates. Instead of the edge device continuously running both models or performing all detections locally, the system extracts the heavy computational workload to the cloud while the edge device maintains only the lightweight model and update mechanisms. This extraction reduces edge computing resource usage while preserving accuracy through cloud-based refinement.
Solution Approach 2:
The patent applies discarding and recovering by selectively applying cloud model detections to the edge device cache only when necessary. The system discards redundant real-time processing at the edge and recovers accuracy through asynchronous updates from the cloud. This approach eliminates the need for continuous redundant computations at the edge while maintaining high accuracy through selective cloud-based refinement.
Data Source
AI summary
Asynchronously updating object detections within a video stream. A first set of objects associated a first frame include a first object detected by a first detection model. Object detection is initiated on a second frame by a second detection model. A second set of objects are identified as being associated with a third frame that is subsequent to the first frame in the video stream. The first object is included in the second set based on tracking the first object from the first frame to the third frame. A second object is identified within the second frame based on the second detection model. When the first object corresponds to the second object but has a different attribute, an attribute of the first object is updated. When the first object does not correspond to the second object, the second object is fast-tracked into the third frame.