Memory management method and device for neural network reasoning
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A neural network and memory management technology, applied in the direction of reasoning methods, biological neural network models, multi-programming devices, etc.
Active Publication Date: 2021-01-22
SENSLAB INC
View PDF9 Cites 1 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
[0010] The technical problem solved by the present invention is: how to reduce the memory occupation of neura
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0087] As described below, an embodiment of the present invention provides a memory management method for neural network reasoning.
[0088] refer to figure 1 The flow chart of the memory management method for neural network reasoning is shown in detail below through specific steps:
[0089] S101. Divide a memory space, where the divided memory space includes at least a first-type area and a second-type area.
[0090] Wherein, the first type of area can only be used to store FM data with a life cycle of 1, and the second type of area can be used to store FM data with any life cycle.
[0091] In other embodiments, the divided memory space includes at least one of the first type of area and the second type of area.
[0092] The writing method of the first type of area is:
[0093]This memory area is used to store FM data with a lifetime of only 1, that is, the OFM data of the nth layer of the neural network will only be used by its next layer, so the network will start to cal...
Embodiment 2
[0170] As described below, an embodiment of the present invention provides a memory management device for neural network reasoning.
[0171] The memory management device of the neural network reasoning comprises:
[0172] a processor adapted to load and execute instructions of a software program;
[0173] A memory adapted to store a software program comprising instructions for performing the following steps:
[0174] Divide the memory space, and the divided memory space includes at least the first type of area and the second type of area; wherein, the first type of area will only be used to store FM data with a lifetime of 1, and the second type Regions can be used to store FM data of any life cycle;
[0175] Analyze the neural network to be allocated memory space to obtain the number of layers with multiple inputs in the neural network;
[0176] Determine whether to enable the first type of area and whether to enable the second type of area according to the number of layer...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention discloses a memory management method and device for neural network reasoning, and the method comprises the steps: dividing a memory space into a first type of region and a second type ofregion, enabling the first type of region to be only used for storing FM data with the life cycle of 1, and enabling the second type of region to be used for storing FM data with any life cycle; analyzing the neural network to which the memory space is to be allocated, and determining whether to start a first type of region and a second type of region according to the number of layers with multiple inputs in the neural network and the total number of layers of the neural network; and allocating a memory space to the FM data of each layer in the neural network from the first type of region and/or the second type of region. According to the method, a proper memory management strategy is adaptively selected according to the structure of the neural network, and memory use is optimized. According to the method, a greedy algorithm is used for searching for the optimal memory allocation scheme layer by layer, memory occupation of neural network reasoning can be reduced, and memory usage is minimized as much as possible.
Description
technical field [0001] The invention relates to the technical field of artificial intelligence, in particular to a memory management method and device for neural network reasoning. Background technique [0002] Thanks to the efficiency and accuracy of deep neural networks, especially in tasks such as detection, recognition and classification, the application of deep neural networks in daily life has continued to expand and diverge in recent years. As a result, various embedded neural network processors (NPUs) emerged as the times require. [0003] However, deep neural networks usually occupy a large amount of memory, which increases the requirements for hardware and directly leads to an increase in the production cost of hardware. Therefore, how to reduce the memory usage of the deep neural network is an urgent problem to be solved at present, which can greatly reduce the hardware requirements of the deep neural network and save costs. [0004] The existing neural network ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more