Competitive resource allocation method and device, electronic equipment and storage medium

Abstract

The invention provides a competitive resource allocation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a task demand submitted by a user; wherein the task demand is a hybrid parallel training resource demand; processing the task demand based on a preset greedy strategy to generate a hybrid parallel resource allocation scheme; and under the condition of determining to generate the hybrid parallel resource allocation scheme, executing the hybrid parallel resource allocation scheme to obtain a competitive resource allocation result. According to the competitive resource allocation method and device, the electronic equipment and the storage medium provided by the invention, the defect that the communication overhead of cross-physical node interaction between the training containers is increased in the prior art can be solved, and the communication overhead between the training containers is reduced.

Description

technical field [0001] The present invention relates to the technical field of resource allocation, and in particular, to a competitive resource allocation method, device, electronic device and storage medium. Background technique [0002] In the field of distributed training, the scale of data to be trained and the scale of parameters for training models may be large. Therefore, there are two distribution modes, data parallelism and model parallelism. Data parallelism is a common way to solve big data problems by distributing data to different machines for computation. Model parallelism is a way of dividing large-scale deep learning models and assigning them to multiple machines for computation. [0003] Due to the large scale of model parameters and data in large-scale deep learning tasks, the application of data parallelism or training model parallelism alone is often insufficient to achieve the expected distributed training effect. Consider using data parallelism plus ...

AI Technical Summary

Problems solved by technology

[0007] The present invention provides a competitive resource allocation method, device, electronic equipment, and storage medium, which are used to solve the defect of increasing communication overhead of cross-physical node interaction between training containers in the prior art, and reduce communication overhead between training containers

Images