Neural Network Memory Allocation for Fragmentation Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for allocating memory in artificial neural networks often result in suboptimal use of global memory area, leading to excessive fragmentation and increased memory size, particularly in integrated circuits with limited memory, and are not adaptable for neural networks where intermediate data is shared among multiple layers.
Innovation Solution
A method that determines optimal placements for intermediate data blocks in the global memory area by exploring multiple sequences of placements, prioritizing certain placements, and dynamically adjusting the memory size to minimize the required memory, allowing for efficient use of integrated circuits with small memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If intermediate data blocks are placed only in free areas of the global memory area, then memory allocation is simple, but excessive memory fragmentation occurs leading to wasted memory
Solution Approach 1:
The patent applies preliminary action by pre-defining multiple possible placements for each intermediate data block before execution. The allocation method pre-calculates optimal placement sequences that minimize fragmentation, rather than allocating memory ad-hoc during runtime. This prevents memory waste by anticipating and avoiding fragmentation issues before they occur.
Solution Approach 2:
The patent introduces dynamics by allowing the memory allocation strategy to adapt based on the specific neural network architecture and execution pattern. The system dynamically selects from multiple predefined placement sequences, choosing the optimal one based on actual memory usage patterns rather than following a static allocation rule.
2Quantity of substance
If superposition allocation is used to reduce memory size, then memory usage is optimized, but it only works for sequential-type neural networks where intermediate data can be overwritten
Solution Approach 1:
The patent achieves universality by creating a memory allocation method that works for both sequential-type neural networks and non-sequential types. The system defines multiple possible placements for each intermediate data block and selects sequences that respect data dependencies regardless of network architecture. This allows the same allocation method to be applied universally across different neural network types while still optimizing memory usage where possible.
Solution Approach 2:
The patent applies parameter changes by varying the placement strategy based on the specific characteristics of the neural network. For sequential networks, it can use aggressive overwriting; for non-sequential networks, it adjusts to use multiple placements or preserve data longer. The allocation parameters are dynamically adjusted based on the network's data flow patterns.
3Quantity of substance
If multiple placement sequences are explored to minimize global memory area size, then memory optimization is improved, but computational complexity increases
Solution Approach 1:
The patent reduces runtime complexity by performing the computationally intensive sequence exploration in advance, during the definition phase. Multiple possible placements and their sequences are pre-calculated and stored. During actual execution, the system simply selects from these pre-defined sequences based on the specific neural network being executed, avoiding the need to perform complex optimization calculations at runtime.
Data Source
AI summary
According to one aspect, a method for determining, for a memory allocation, placements in a memory area of data blocks generated by a neural network, comprises a development of an initial sequence of placements of blocks, each placement being selected from several possible placements, the initial sequence being defined as a candidate sequence, a development of at least one modified sequence of placements from a replacement of a given placement of the initial sequence by a memorized unselected placement, and, if the planned size of the memory area obtained by this modified sequence is less than that of the memory area of the candidate sequence, then this modified sequence becomes the candidate sequence, the placements of the blocks for the allocation being those of the placement sequence defined as a candidate sequence once each modified sequence has been developed.