Neural Network Memory Allocation with Size-Based Block Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory allocation methods for neural networks result in inefficient use of memory resources due to the allocation and reuse of memory blocks based on sequential requests, leading to excessive memory occupation and improper allocation.
Innovation Solution
A memory allocation method that sorts memory blocks by size and optimizes allocation based on a global reuse solution, ensuring that allocated memory blocks are reused for larger blocks and new blocks are allocated for smaller ones, thereby reducing overall memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If memory blocks are allocated and reused based on sequential requests, then the allocation process is simple and fast, but the total memory occupation increases and memory efficiency deteriorates
Solution Approach 1:
The patent applies preliminary action by sorting all memory blocks in advance based on their size and usage patterns before allocation. This pre-processing step organizes memory blocks in descending order of size, enabling the allocation algorithm to systematically assign larger blocks first and identify reuse opportunities more effectively, thereby reducing total memory occupation while maintaining operational simplicity
Solution Approach 2:
The patent changes the allocation parameter from sequential request order to size-based sorted order. By reordering memory blocks according to their size parameters, the system optimizes the allocation strategy to allocate larger blocks first and reuse them more effectively, transforming the allocation process from a simple sequential approach to an optimized size-based approach that reduces overall memory usage
2Productivity
If a large memory block is reused for a small memory block, then allocation speed is maintained, but the large memory block cannot be reallocated for its original purpose, reducing memory versatility
Solution Approach 1:
The patent applies local quality by implementing size-matched memory block allocation where each memory block is allocated to tasks requiring similar size characteristics. By sorting and matching blocks of comparable sizes, the system ensures that large blocks remain available for large tasks while small blocks serve small tasks, maintaining both allocation speed and the versatility to reallocate blocks appropriately for their original purposes
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides a memory allocation method and apparatus for a neural network, where N memory blocks need to be occupied during running of the neural network, and the method includes: determining information about the N memory blocks; sorting the N memory blocks in descending order based on sizes of the N memory blocks; allocating a first memory block to the neural network, to perform an operation for a first node set in the neural network; determining whether a memory block in an allocated set can be reused for an nth memory block, where the allocated set includes a memory block that has been allocated to the neural network; and if the memory block in the allocated set can be reused for the nth memory block, allocating, to the neural network, the memory block that can be reused, to perform an operation for an nth node set in the neural network, and then updating the allocated set; and sequentially performing the foregoing determining from n = 2 to n = N based on the sort sequence. According to the method and apparatus in this application, memory allocation of the neural network can be optimized.