Edge DNN Workload Scheduling for Multi-Camera Core Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing edge devices lack efficient scheduling frameworks to support multiple camera streams and DNN models, leading to suboptimal resource utilization and performance degradation, especially when handling diverse DNN architectures and frameworks.
Innovation Solution
A framework that leverages insights about DNN structures to efficiently schedule workloads, using a profiler to estimate resource requirements and an allocator to assign cores and parameters, ensuring no DNN workload is starved, even with increasing workloads, and supporting various DNN models and camera streams.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple DNN models are run on multiple camera streams simultaneously, then the system functionality and versatility are improved, but the resource consumption and processing complexity increase beyond the capacity of typical edge devices
Solution Approach 1:
The patent segments DNN workloads into multiple independent tasks that can be scheduled and executed on different CPU cores simultaneously. Each camera stream and DNN model combination is treated as a separate schedulable task, allowing the system to manage complexity through modular task decomposition rather than monolithic processing.
Solution Approach 2:
The patent creates a universal scheduling framework that can handle diverse DNN models (e.g., YOLO, SSD, Faster R-CNN) and multiple camera streams through a single multi-core processor. The framework provides multi-functionality by supporting various DNN architectures and frameworks (TensorFlow, PyTorch, Caffe) without requiring dedicated hardware for each model type.
2Speed
If dedicated cores are assigned to each DNN on each camera stream, then the processing speed and frame rate are improved, but the hardware resources required exceed what is available on typical edge devices
Solution Approach 1:
The patent implements dynamic core allocation where CPU cores are not permanently dedicated to specific DNN tasks but are dynamically assigned based on current workload demands. The scheduler monitors task priorities, deadlines, and resource availability to flexibly allocate cores across multiple DNN inference tasks, enabling high frame rates without requiring a dedicated core for each task.
Solution Approach 2:
The patent merges multiple DNN inference tasks onto shared CPU cores through efficient scheduling. Multiple camera streams and DNN models share the same hardware resources (GPUs and CPUs) by time-multiplexing and parallel processing across available cores, reducing the total hardware footprint while maintaining acceptable performance levels.
3Loss of time
If compute-heavy DNN workloads are run locally on edge devices, then latency is reduced and cloud bandwidth consumption decreases, but the limited GPU and CPU resources of edge devices become insufficient
Solution Approach 1:
The patent ensures continuous processing of multiple camera streams by implementing a scheduling framework that maintains steady throughput across all active DNN tasks. The scheduler prevents idle time between inference operations and ensures that CPU cores are continuously utilized for productive DNN processing, maximizing the limited edge device resources to achieve acceptable latency without overwhelming the processing capacity.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Systems, methods, and computer-executable instructions for scheduling neural network workloads on an edge device. A performance model for each neural network model is received. Parameters for each neural network workload is determined based on an associated performance model. Processing core assignments are determined from a plurality of processing cores for each neural network workload based on the corresponding performance model and processing core utilization. Image streams are received and associated with a neural network workload. Each neural network workload is scheduled to run on the processing cores based on the processing core assignments.