Method and device for dispatching HPC (high performance computing) cluster work

Abstract

The invention provides a method for dispatching a HPC (high performance computing) cluster work and a device for dispatching the HPC cluster work. The method comprises the following steps: setting a grouping rule, setting a grouping sequencing rule, and setting conditions for judging the grouping; according to the set grouping rule, dividing all works in the HPC cluster into multiple groups; according to the set grouping sequencing rule, sequencing multiple groups; according to multiple groups after sequencing, orderly polling every group, and taking out one work from groups satisfied with the setting condition to execute; after all groups are polled, repeatedly executing the step until all groups cannot meet the setting conditions; when remaining calculation resources in the HPC cluster are restored until the group can meet the setting condition; sequencing according to the group and taking out one work from the groups satisfied with the setting conditions to execute until all works are finished. The method and the device for dispatching the HPC cluster work can realize the reasonable distribution of the HPC cluster calculation resource.

Description

technical field [0001] The invention relates to data processing technology, in particular to an HPC cluster job scheduling method and device. Background technique [0002] HPC (High Performance Computing, High Performance Computing) cluster is a group of servers connected through a high-performance network, which can work together to complete data analysis or scientific computing tasks. With the widespread application of cluster systems in scientific computing and business operations, the role of cluster systems is becoming more and more important, and has gradually become an indispensable tool in scientific computing and business services. More and more users use clusters to complete computing tasks. The same cluster will be used by hundreds of users at the same time, and each user will submit multiple computing tasks. When the computing resources of the cluster are insufficient, these jobs will be in a waiting state. [0003] For these queued tasks, it is necessary to uni...

AI Technical Summary

Problems solved by technology

However, in some cases, the above method cannot allocate resources well. For example, user A submits hundreds of jobs at the same time. These jobs are queued for processing and occupy the computing resources of the cluster. No resources can be allocated until all the tasks of user A are completed before the job of user B can be executed, which leads to unreasonable resource allocation

Images