Deep Learning Autotuning Scheduling for Resource Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autotuning frameworks for deep learning models face inefficiencies in resource utilization and completion time due to sequential and monolithic design, leading to idle resources and performance degradation when run on different computation resources.
Innovation Solution
A redesigned autotuning framework decomposes the process into functional sub-procedures and schedules them simultaneously, using a scheduler with a Shortest Job First policy and Multi-Process Service capability to ensure exclusive resource access and optimize resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If autotuning tasks are executed sequentially on different computation resources, then resource isolation and correct measurement results are ensured, but total autotuning completion time increases and resource utilization decreases
Solution Approach 1:
The patent segments the autotuning process into independent sub-tasks that can be executed in parallel. Each sub-task is assigned to a specific computation resource with guaranteed exclusive access, allowing simultaneous execution across multiple resources while maintaining measurement accuracy through proper isolation.
Solution Approach 2:
The patent implements dynamic task scheduling that assigns autotuning tasks to computation resources based on current resource availability and task characteristics. This dynamic allocation enables parallel execution when resources are available while ensuring exclusive access when resources are contested, optimizing both completion time and measurement accuracy.
2Productivity
If multiple autotuning tasks share computation resources simultaneously, then resource utilization improves and completion time decreases, but resource contention causes performance degradation and incorrect measurements
Solution Approach 1:
The scheduling system dynamically determines resource allocation based on task state and resource availability. When computation resources are available, multiple tasks execute in parallel with high resource utilization. When resources are contested, the system provides exclusive access to ensure measurement accuracy, thus adapting to conditions in real-time.
Solution Approach 2:
The system implements self-service mechanisms where the scheduler automatically manages resource allocation and task distribution without external intervention. This enables the system to autonomously optimize resource utilization while maintaining measurement integrity through built-in conflict resolution and exclusive access protocols.
3Device complexity
If a simple scheduling policy is used, then implementation complexity is reduced, but resource utilization is insufficient and completion time increases
Solution Approach 1:
The scheduling policy implements self-service through automatic task assignment and resource allocation based on predefined rules and current system state. This automated approach achieves high resource utilization without requiring complex manual scheduling or intensive computational optimization, balancing simplicity with effectiveness.
Data Source
AI summary
Systems and methods are provided for improving autotuning procedures. For example, the system can implement a task launcher, a scheduler, and an agent to launch, schedule, and execute decomposed autotuning stages, respectively. The scheduling policy implemented by the scheduler may perform operations beyond a simple scheduling policy (e.g., a FIFO-based scheduling policy), which produces a high queuing delay. By leveraging autotuning specific domain knowledge, this may help reduce queuing delay and improve resource utilization that is otherwise found in traditional systems.