Deep learning memory management method and system based on tensor access

Abstract

The present invention provides a deep learning memory management method based on Tensor access. The method obtains the memory space overhead and time overhead under relevant decisions by collecting the execution information of the neural network and the performance information of the hardware platform, and establishes an integer linear programming model. By optimizing and solving the optimal Tensor scheduling strategy under constraints, it can solve the problem of insufficient memory and obtain high deep learning training performance. Compared with the prior art, under the same hardware performance, the present invention can realize neural network training with a larger batch size. The present invention also proposes a memory management system, including a profile module, a decision-making module and an execution module; the system can be directly added to the deep learning framework and is easy to use.

Description

technical field [0001] The present invention relates to the field of computer science and artificial intelligence, in particular to a Tensor-based deep learning memory management method and system. Background technique [0002] The innovation of deep learning technology has greatly promoted the development of computer vision, natural language processing, medicine and other fields. Wider and wider network structures have higher accuracy in training, so it has become a trend for the deep learning community to adopt large and wide networks. However, the storage capacity of deep learning accelerators represented by GPUs is very limited and cannot accommodate a large amount of data in the training process of deep learning models, which seriously restricts the development of deep learning technology. The larger and deeper the network structure requires more memory usage, it is of great value to solve the problem of insufficient memory for deep learning training. Existing solutio...

AI Technical Summary

Problems solved by technology

That is, by deciding on Tensor to make certain decisions in deep learning, the problem of insufficient memory can be solved and efficient deep learning training performance can be obtained at the same time.

Images