Deep learning memory management method and system based on Tensor access

Abstract

The invention provides a deep learning memory management method based on Tensor access, and the method comprises the steps: collecting the execution information of a neural network and the performanceinformation of a hardware platform, obtaining the memory space expenditure and time expenditure under a related decision, building an integer linear programming model, solving an optimal Tensor scheduling strategy through the optimization under a constraint condition, and obtaining the memory space expenditure and time expenditure under the related decision, so that the problem of memory insufficiency is solved, and high deep learning training performance is obtained. Compared with the prior art, under the same hardware performance, larger batchsize neural network training can be realized. The invention further provides a memory management system. The memory management system comprises a profile module, a decision module and an execution module. The system can be directly added to a deeplearning framework and is convenient to use.

Description

technical field [0001] The present invention relates to the field of computer science and artificial intelligence, in particular to a Tensor-based deep learning memory management method and system. Background technique [0002] The innovation of deep learning technology has greatly promoted the development of computer vision, natural language processing, medicine and other fields. Wider and wider network structures have higher accuracy in training, so it has become a trend for the deep learning community to adopt large and wide networks. However, the storage capacity of deep learning accelerators represented by GPUs is very limited and cannot accommodate a large amount of data in the training process of deep learning models, which seriously restricts the development of deep learning technology. The larger and deeper the network structure requires more memory usage, it is of great value to solve the problem of insufficient memory for deep learning training. Existing solutio...

AI Technical Summary

Problems solved by technology

That is, by deciding on Tensor to make certain decisions in deep learning, the problem of insufficient memory can be solved and efficient deep learning training performance can be obtained at the same time.

Images