Edge Video Analytics Resource Scheduling for Data-Drift Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Edge computing devices face challenges in maintaining high inference accuracy due to data drift and resource constraints, particularly when using compressed deep neural networks (DNNs) for video analytics, as they are vulnerable to varying scenes and object distributions over time, and continuous retraining introduces a tradeoff between live inference accuracy and model improvement.
Innovation Solution
A micro-profiler estimates the benefits and costs of retraining configurations, and a scalable resource scheduler allocates computing resources between retraining and inference jobs on edge servers, using a thief scheduler to prioritize resource allocation and hyperparameter configurations that maintain high inference accuracy while minimizing resource waste.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If continuous retraining is performed to maintain high inference accuracy, then model accuracy is improved, but computing resources are consumed
Solution Approach 1:
The patent implements dynamic resource allocation where the system continuously monitors inference accuracy metrics and automatically adjusts retraining schedules and resource allocation. The scheduler dynamically determines when retraining is necessary based on actual performance degradation from data drift, rather than following a fixed retraining schedule, thus optimizing the balance between accuracy maintenance and resource consumption.
Solution Approach 2:
The system changes hyperparameters such as batch size, learning rate, and retraining frequency based on observed performance metrics and available resources. The micro-profiler adjusts these parameters dynamically to achieve optimal accuracy improvement per unit of computing resource, allowing the model to adapt to data drift efficiently without wasting resources on unnecessary retraining iterations.
2Reliability
If more computing resources are allocated to retraining, then model accuracy is improved, but resources available for live inference are reduced
Solution Approach 1:
The system performs preliminary profiling and evaluation to determine the optimal retraining configuration before executing full retraining. The micro-profiler conducts quick assessments to identify when retraining is actually needed and what configuration would be most effective, preventing unnecessary resource allocation to retraining that would disrupt live inference operations.
Solution Approach 2:
The system automatically monitors its own performance metrics and self-manages the retraining process without external intervention. The scheduler detects when accuracy degradation occurs and autonomously allocates resources for retraining, then automatically restores resources to inference operations once retraining is complete, minimizing the impact on continuous inference throughput.
3Productivity
If compressed DNNs are used to reduce resource requirements, then device efficiency is improved, but vulnerability to data drift increases
Solution Approach 1:
The system implements periodic retraining cycles for compressed DNNs based on monitored performance degradation. Rather than continuous retraining, the system periodically updates the model when data drift is detected, allowing the compressed model to maintain robustness against data drift while minimizing resource consumption and maintaining device efficiency between retraining cycles.
Solution Approach 2:
The system uses a composite approach combining compressed DNN architecture with selective retraining mechanisms. The compressed model provides efficient inference on edge devices, while the periodic retraining process using accumulated data restores and maintains robustness to data drift, creating a hybrid solution that achieves both efficiency and reliability.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Examples are disclosed that relate to methods and computing devices for allocating computing resources and selecting hyperparameter configurations during continuous retraining and operation of a machine learning model. In one example, a computing device configured to be located at a network edge between a local network and a cloud service comprises a processor and a memory storing instructions executable by the processor to operate a machine learning model. During a retraining window, a selected portion of a video stream is selected for labeling. At least a portion of a labeled retraining data set is selected for profiling a superset of hyperparameter configurations. For each configuration of the superset of hyperparameter configurations, a profiling test is performed. The profiling test is terminated, and a change in inference accuracy that resulted from the profiling test is extrapolated. Based upon the extrapolated inference accuracies, a set of selected hyperparameter configurations is output.