Gpu cluster scheduling method for training and inference integrated deep learning

Abstract

The application discloses a GPU cluster scheduling method for training and reasoning integrated deep learning, can realize mixed deployment of online business of reasoning and offline business of training, improves cluster resource utilization and reduces training cost of training task under the premise of guaranteeing service quality of reasoning task. The application fully considers challenges under mixed deployment of training and reasoning task, uses a low-complexity knapsack algorithm to model and schedule training task, and saves high scheduling cost of genetic algorithm under large-scale cluster and load conditions; reasoning scheduling algorithm is used to manage training and reasoning cluster computing resources, and influence of training and reasoning task in training and reasoning cluster resource lending and recycling stages is reduced; adaptive elastic training is introduced, and on the basis of fast elastic training, a hyperparameter optimization link is introduced, so that the overall execution time of the training task is reduced under the condition that the additional cost is negligible.

Images