Cluster resource scheduling method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a cluster resource scheduling method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a resource scheduling request for a GPU in a GPU cluster in a first operation environment; executing a task scheduling strategy according to the request parameter to add the deep learning task into a task queue, and executing a preset resource allocation strategy to determine at least one target GPU from a GPU cluster; scheduling the deep learning task to at least one target GPU for processing; and adjusting the task scheduling strategy and the preset resource allocation strategy, and deploying the adjusted task scheduling strategy and the preset resource allocation strategy in the second operation environment. The embodiment of the invention is favorable for reducing the development cost of a resource scheduling algorithm.

Description

technical field [0001] The present application relates to the technical field of distributed systems, in particular to a cluster resource scheduling method and device, electronic equipment and storage media. Background technique [0002] With the development of artificial intelligence, deep learning has become the focus of researchers, and it has a wide range of applications in target recognition and target detection tasks. The implementation of deep learning algorithms is inseparable from effective training. In order to meet the computing power requirements of its training, large-scale GPU (graphics processing unit, graphics processing unit) clusters have become the support for the research and development of deep learning algorithms. When it comes to clusters, it will inevitably involve resource management and task scheduling. At present, task scheduling in clusters mostly depends on task scheduling algorithms. After the task scheduling algorithm is developed, it needs to ...

AI Technical Summary

Problems solved by technology

When it comes to clusters, it will inevitably involve resource management and task scheduling. At present, task scheduling in clusters mostly depends on task scheduling algorithms. After the task scheduling algorithm is developed, it needs to be tested in the cluster to verify its effectiveness. and reliability, but in terms of the characteristics of long deep learning training cycle and high computing density, the development and deployment process of the current task scheduling algorithm is time-consuming and laborious, which makes the development cost high

Images