Cloud GPU Video memory scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a cloud GPU video memory scheduling method and device, electronic equipment and a storage medium, and the method comprises the steps: receiving an operation request sent by a terminal, wherein the operation request comprises an algorithm program of a deep learning network model and to-be-learned data; according to the algorithm program and the to-be-learned data, determiningthe actual video memory capacity required by the operation of the deep learning network model; and distributing a GPU for the deep learning network model from a GPU pool according to the actual videomemory capacity. According to the embodiment of the invention, when the GPU is allocated to the deep learning network model, the actual video memory capacity required by the operation of the deep learning network model is calculated according to the network structure information, the data dimension information, the batch processing quantity and other information of the deep learning network model. The GPU is allocated to the deep learning network model according to the actual video memory capacity, it can be ensured that the adaptation degree of the allocated GPU is higher, the GPU video memory scheduling efficiency is improved, and the utilization rate of a GPU pool can be increased.

Description

technical field [0001] The application belongs to the technical field of deep learning, and in particular relates to a method, device, electronic device and storage medium for video memory scheduling of cloud GPU. Background technique [0002] With the increase in the amount of deep learning training business data, the GPU (Graphics Processing Unit, Graphics Processing Unit) memory capacity required for deep learning network models increases, and the cost of configuring GPUs for users is very high, so many users choose to use the GPU pool in the cloud to run deep learning network models. The cloud needs to allocate a suitable GPU for the requested deep learning network model. [0003] In related technologies, a distributed computing power scheduling method for deep learning network models is proposed. The GPU pool on which this method is based has less memory capacity for each GPU, and a single GPU cannot run the deep learning network model normally. Scheduling allocates m...

AI Technical Summary

Problems solved by technology

[0004] However, the above-mentioned related technologies are not applicable to GPU pools where each GPU has a large memory capacity. In such a GPU pool, a single GPU can provide the video memory required by the de

Images