Operand-Based Task Scheduling for Deep Learning Dependency Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current task scheduling methods for deep learning model training are inefficient due to their reliance on Directed Acyclic Graphs (DAG) for managing dependency relationships among operation tasks, which does not optimize scheduling effectively.
Innovation Solution
A task scheduling method that determines dependency relationships among operation tasks based on their operands, allowing for fine-grained scheduling by minimizing dependencies and enabling parallel execution of tasks without dependencies, while merging communication operations to improve communication efficiency and prioritizing memory reclaim operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If task scheduling is based on DAG to manage dependency relationships, then the training process can proceed according to a correct procedure, but the scheduling efficiency is poor
Solution Approach 1:
The patent segments the task scheduling process by introducing a priority queue mechanism that divides tasks into different priority levels. Tasks are segmented based on their urgency and dependency relationships, allowing the system to process critical tasks first while maintaining correct procedural execution through priority-based ordering.
Solution Approach 2:
The patent introduces a priority queue as an intermediary data structure between the DAG dependency management and the actual task execution. This intermediary layer enables efficient task selection and scheduling by mediating between the rigid DAG structure and the flexible execution requirements, thus improving scheduling efficiency without compromising procedural correctness.
2Productivity
If fine-grained scheduling is implemented by determining dependency relationships based on operands, then task dependency is minimized and scheduling is improved, but the complexity of dependency analysis increases
Solution Approach 1:
The patent applies local quality by analyzing dependency relationships at the operand level rather than at the task level. This localized analysis approach examines specific data dependencies between operations, allowing fine-grained scheduling decisions to be made based on actual data flow requirements, thus minimizing unnecessary task dependencies while keeping the analysis manageable through focused scope.
Solution Approach 2:
The patent changes the parameter of dependency analysis from task-level to operand-level. By shifting the analysis granularity to the operand level, the system can identify more precise dependency relationships, enabling better parallelization opportunities while managing complexity through the use of standardized operand comparison operations.
3Loss of energy
If communication operations are merged to improve communication efficiency, then communication overhead is reduced, but the scheduling flexibility is reduced
Solution Approach 1:
The patent merges communication operations by combining multiple communication tasks into single consolidated communication calls when they share common characteristics such as target devices or data patterns. This merging reduces communication overhead and improves efficiency while the priority queue mechanism maintains scheduling flexibility by allowing dynamic adjustment of merged task priorities based on changing system conditions.
Solution Approach 2:
The patent introduces dynamics by making the communication merging strategy adaptive rather than static. The system dynamically determines whether to merge communication operations based on real-time conditions such as task priorities, resource availability, and dependency relationships. This dynamic approach allows the system to optimize communication efficiency when conditions permit while maintaining scheduling flexibility when conditions require more granular control.
Data Source
AI summary
Provided are a task scheduling method and device, and a computer storage medium. The task scheduling method includes that: a dependency relationship among multiple operation tasks is determined according to operands corresponding to multiple operation tasks in an operation task queue; and the multiple operation tasks in the operation task queue are scheduled based on the dependency relationship among the multiple operation tasks.