Memory management method and device for neural network reasoning

Abstract

The invention discloses a memory management method and device for neural network reasoning, and the method comprises the steps: dividing a memory space into a first type of region and a second type ofregion, enabling the first type of region to be only used for storing FM data with the life cycle of 1, and enabling the second type of region to be used for storing FM data with any life cycle; analyzing the neural network to which the memory space is to be allocated, and determining whether to start a first type of region and a second type of region according to the number of layers with multiple inputs in the neural network and the total number of layers of the neural network; and allocating a memory space to the FM data of each layer in the neural network from the first type of region and / or the second type of region. According to the method, a proper memory management strategy is adaptively selected according to the structure of the neural network, and memory use is optimized. According to the method, a greedy algorithm is used for searching for the optimal memory allocation scheme layer by layer, memory occupation of neural network reasoning can be reduced, and memory usage is minimized as much as possible.

Description

technical field [0001] The invention relates to the technical field of artificial intelligence, in particular to a memory management method and device for neural network reasoning. Background technique [0002] Thanks to the efficiency and accuracy of deep neural networks, especially in tasks such as detection, recognition and classification, the application of deep neural networks in daily life has continued to expand and diverge in recent years. As a result, various embedded neural network processors (NPUs) emerged as the times require. [0003] However, deep neural networks usually occupy a large amount of memory, which increases the requirements for hardware and directly leads to an increase in the production cost of hardware. Therefore, how to reduce the memory usage of the deep neural network is an urgent problem to be solved at present, which can greatly reduce the hardware requirements of the deep neural network and save costs. [0004] The existing neural network ...

AI Technical Summary

Problems solved by technology

[0010] The technical problem solved by the present invention is: how to reduce the memory occupation of neural network reasoning, to reduce hardware consumption and the hardware cost required for running neural network

Images