Cloud GPU Video memory scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a cloud GPU video memory scheduling method and device, electronic equipment and a storage medium, and the method comprises the steps: receiving an operation request sent by a terminal, wherein the operation request comprises an algorithm program of a deep learning network model and to-be-learned data; according to the algorithm program and the to-be-learned data, determiningthe actual video memory capacity required by the operation of the deep learning network model; and distributing a GPU for the deep learning network model from a GPU pool according to the actual videomemory capacity. According to the embodiment of the invention, when the GPU is allocated to the deep learning network model, the actual video memory capacity required by the operation of the deep learning network model is calculated according to the network structure information, the data dimension information, the batch processing quantity and other information of the deep learning network model. The GPU is allocated to the deep learning network model according to the actual video memory capacity, it can be ensured that the adaptation degree of the allocated GPU is higher, the GPU video memory scheduling efficiency is improved, and the utilization rate of a GPU pool can be increased.

Description

Technical field [0001] This application belongs to the field of deep learning technology, and specifically relates to a cloud GPU memory scheduling method, device, electronic equipment and storage medium. Background technique [0002] As the amount of deep learning training business data increases, the GPU (Graphics Processing Unit, graphics processor) memory capacity required for deep learning network models becomes larger. It is very expensive for users to equip their own GPUs, so many users choose to use cloud GPU pools. to run the deep learning network model. The cloud needs to allocate appropriate GPUs for the deep learning network models requested to be run. [0003] In related technology, a distributed computing power scheduling method for deep learning network models is proposed. The GPU pool on which this method is based has less video memory capacity for each GPU, and a single GPU cannot run the deep learning network model normally. Therefore, through distribution...

AI Technical Summary

Problems solved by technology

[0004] However, the above-mentioned related technologies are not applicable to GPU pools where each GPU has a large memory capacity. In such a GPU pool, a single GPU can provide the video memory required by the deep learning network model submitted by the user. Therefore, in such a GPU pool Among them, the distributed computing power scheduling method in related technologies cannot achieve efficient video memory scheduling

Images