Optimizing Compute Resources for Distributed Batch Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed batch optimization jobs, such as Deep Learning jobs, face inefficiencies due to slow communication networks and inadequate resource allocation, leading to prolonged training times and increased costs, as additional computing nodes do not always provide proportional benefits.
Innovation Solution
The method determines optimal compute resources by calculating node counts and computing resource configurations based on model size, batch sizes, and communication overhead, utilizing a large batch simulation mode to minimize communication time and cost, by estimating execution times and costs for varying node counts and selecting the most efficient configuration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If additional computing nodes are added to process distributed batch optimization jobs, then processing capacity increases, but communication time increases and cost increases without proportional benefit
Solution Approach 1:
The patent changes the batch size parameter to be larger when using fewer computing nodes, and adjusts the node count parameter accordingly. By varying these parameters together, the system optimizes the balance between processing capacity and communication time, achieving efficient processing without adding excessive nodes that would increase communication overhead
Solution Approach 2:
The system dynamically determines the optimal number of computing nodes based on the specific job requirements, model size, and data characteristics rather than using a fixed or linearly scaled node configuration. This dynamic adjustment allows the system to adapt node count to actual needs, avoiding unnecessary communication overhead
2Productivity
If additional computing nodes are added to process distributed batch optimization jobs, then processing capacity increases, but cost increases without proportional benefit
Solution Approach 1:
The patent optimizes the cost parameter by changing the node count and batch size parameters together. The system calculates the total cost as a function of node count and execution time, then selects parameter combinations that minimize cost while meeting processing requirements, avoiding the wasteful expenditure of adding excessive computing nodes
Solution Approach 2:
The system performs self-optimization by automatically determining the optimal computing resource configuration without requiring manual intervention or over-provisioning. The resource determination module autonomously selects the most cost-effective node count and batch size combination based on job characteristics
3Loss of time
If batch size is increased to reduce communication frequency, then communication time decreases, but computation time per batch increases
Solution Approach 1:
The patent simultaneously adjusts the batch size parameter and node count parameter to optimize the trade-off between communication time and computation time per batch. By changing both parameters together rather than independently, the system achieves a balanced configuration where communication overhead is reduced without creating excessive computation burden on individual nodes
Data Source
AI summary
Methods, systems, and computer program products for determining optimal compute resources for distributed batch based optimization applications are provided herein. A method includes obtaining a size of an input dataset, a size of a model, and a set of batch sizes corresponding to a job to be processed using a distributed computing system; computing, based at least in part on the set of batch sizes, one or more node counts corresponding to a number of nodes that can be used for processing said job; estimating, for each given one of the node counts, an execution time to process the job based on an average computation time for a batch of said input dataset and an average communication time for said batch of said input dataset; and selecting, based at least in part on said estimating, at least one of said node counts for processing the job.