Cluster training node allocation method and electronic equipment

Abstract

The invention relates to the field of node distribution, in particular to a cluster training node distribution method and electronic equipment. The method comprises the steps: setting training distribution parameters, and submitting a cluster training task; obtaining an available node set M, obtaining all grouping information lists and node lists corresponding to selected request group names, and obtaining a request node set Y; obtaining a pre-allocated available node set Z = M intersection Y; checking whether the number of nodes in the pre-allocated available node set Z meets the number of request nodes or not; if yes, establishing an allocation execution node set with the capacity as the number of request nodes; putting the specified nodes into a distribution execution node set; sorting nodes in the pre-allocated available node set Z according to priorities, and sequentially putting the nodes into an allocation execution node set until the allocation execution node set is full; locking and distributing execution nodes, distributing tasks, and starting task training. The method has the advantages that the node distribution mode is optimized, equipment resources are fully utilized, and the overall error report rate and the fault rate of training tasks are reduced.

Description

technical field [0001] The invention relates to the field of node allocation, in particular to a cluster training node allocation method and electronic equipment. Background technique [0002] With the development of AI technology, many complex AI program models need to be trained on computing clusters. Due to the simultaneous use of cluster node resources by multiple users, during the training process, for a single-machine or multi-machine training task, the resources of the node computers need to be allocated. [0003] The allocation of node computers in the prior art is unidirectional and static, either the user directly selects a training group to train the node computers, or the system directly allocates according to the remaining node resources. And unless the user actively disables a node with many error reports, the node allocation will generally not be adjusted according to the running status. This one-way static allocation method is difficult to achieve dynamic op...

AI Technical Summary

Problems solved by technology

And unless the user actively disables a node with many error reports, the node allocation will generally not be adjusted according to the running status. This one-way static allocation method is difficult to achieve dynamic optimization of computing resource allocation, so that resources in good status can be fully utilized. return and use

This can easily lead to improper resource allocation, resource overload, more error reports, and poor user experience

Images