FPGA Model Switching Scheduler for Multi-User Task Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional scheduling methods for CPU are not suitable for switching models on FPGA resources, leading to a lack of support for smoothly switching models executed by multiple users on shared FPGA devices, particularly in scenarios with limited FPGA resources.
Innovation Solution
A scheduling device comprising a controller unit to acquire and switch models, a control unit to set up the accelerator for processing, and a scheduler unit to manage tasks based on a queue for each model, ensuring efficient model switching and resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional CPU scheduling methods are used for FPGA model switching, then scheduling simplicity is maintained, but model switching capability and resource sharing efficiency deteriorate
Solution Approach 1:
The scheduling system segments tasks by model type, creating separate queues for different models (e.g., ResNet50 queue, EfficientNetB0 queue). This allows independent management of each model's tasks while enabling smooth switching between models through the scheduler, which selects tasks from appropriate queues based on current model configuration.
Solution Approach 2:
The scheduling system dynamically adjusts its behavior based on the current model configuration and task requirements. The scheduler can switch between different scheduling strategies (round-robin, priority-based, FIFO) depending on the model type and task characteristics, enabling flexible adaptation to varying workloads while maintaining manageable complexity.
2Productivity
If multiple models share the same FPGA resources, then resource utilization efficiency improves, but scheduling complexity and context management difficulty increase
Solution Approach 1:
The system segments the task management process by creating model-specific queues that organize tasks according to their required model type. This segmentation allows the scheduler to efficiently match tasks to appropriate models without managing all possible model contexts simultaneously, reducing overall system complexity while enabling effective resource sharing.
Solution Approach 2:
The scheduler acts as an intermediary between task submission and model execution. It receives tasks from various sources, determines the appropriate model based on task requirements, and coordinates the switching between models. This intermediary role abstracts the complexity of model context management from the task submission process, enabling efficient resource utilization without exposing users to underlying complexity.
3Reliability
If dedicated FPGA devices are prepared for each model, then model execution performance is optimized, but initial investment and hardware cost increase
Solution Approach 1:
The scheduling system enables a single FPGA device to serve multiple model types through software-based task management. By implementing model-specific queues and intelligent scheduling, the same physical FPGA can efficiently execute different models (e.g., ResNet50, EfficientNetB0) without requiring separate dedicated hardware for each model, achieving multi-functionality that reduces hardware quantity while maintaining execution performance.
Data Source
AI summary
A scheduling device includes a controller unit that acquires a model used by each task, an FPGA control unit that performs control to switch a setting of the FPGA in such a manner that the model acquired by the controller unit becomes processable, and a scheduler unit that refers to a queue that stores a task for each model used by each task, reads a task using a model that has become processable by switching by the FPGA control unit, and causes the FPGA to execute the task.