Neural Network Scheduling via Dynamic Batch Size Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face inefficiencies in scheduling neural networks due to varying memory and processing requirements, which impact the concurrent or parallel execution of neural networks with other applications, leading to suboptimal runtime schedules.
Innovation Solution
A system and method that determine profiles for co-scheduled applications and neural networks, adjust batch sizes based on these profiles, and schedule them accordingly to optimize resource utilization and improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural networks are executed with fixed batch sizes in parallel or concurrent environments, then the scheduling is simple, but the resource utilization is suboptimal and execution efficiency is reduced
Solution Approach 1:
The patent implements dynamic batch size adjustment by determining profiles for co-scheduled applications and neural networks, then varying the batch size based on these profiles to optimize resource utilization. This transforms the static batch size parameter into a dynamic one that adapts to the runtime environment, resolving the contradiction between simple scheduling and optimal efficiency.
Solution Approach 2:
The patent changes the batch size parameter based on determined profiles of co-scheduled applications and neural networks. By adjusting this key parameter dynamically, the system achieves better resource utilization and execution efficiency without requiring fundamentally complex scheduling mechanisms.
2Productivity
If batch size is increased to improve neural network processing throughput, then processing efficiency improves, but memory and processing cycle requirements increase, impacting concurrent application performance
Solution Approach 1:
The patent dynamically adjusts batch size based on determined profiles that characterize the resource requirements and performance characteristics of co-scheduled applications. This allows the system to optimize processing throughput while adapting resource consumption to the actual runtime environment, preventing excessive memory and processing cycle usage.
Solution Approach 2:
The patent changes the batch size parameter based on profile analysis of co-scheduled workloads. By adjusting this parameter according to the specific characteristics of the runtime environment, the system achieves optimal throughput while controlling resource consumption to levels that accommodate concurrent applications.
3Use of energy by moving object
If batch size is decreased to reduce resource consumption, then memory and processing requirements are reduced, but processing throughput and execution efficiency deteriorate
Solution Approach 1:
The patent implements dynamic batch size adjustment that responds to the determined profiles of co-scheduled applications and neural networks. This allows the system to increase batch size when resources are available to maintain high throughput, and decrease it when resources are constrained, optimizing the balance between resource consumption and processing efficiency.
Solution Approach 2:
The patent adjusts the batch size parameter based on profile-based analysis of the runtime environment. This parameter change strategy ensures that batch size is optimized for throughput when resources permit, while being reduced when resource constraints exist, thereby maintaining acceptable performance across varying conditions.
Data Source
AI summary
The present disclosure relates to computer-implemented systems and methods for scheduling a neural network for execution. In one implementation, a system for scheduling a neural network for execution may include at least one memory storing instructions and at least one processor configured to execute the instructions to determine a profile for one or more applications co-scheduled with at least one neural network; determine a batch size for the at least one neural network based on the determined profile for the one or more applications; and scheduling the one or more applications and the at least one neural network based on the batch size.