AI Chip Memory Allocation Using Demand Lifecycles to Cut Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing static memory management methods cause memory fragmentation, reducing memory utilization efficiency in chips that process large amounts of data, such as AI chips.
Innovation Solution
A memory management method that allocates memory space based on sequencing numbers and life cycles of memory demands, using a target memory block to reduce fragmentation and improve utilization efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If static memory management is used, then memory allocation is simpler and faster, but memory fragmentation occurs reducing utilization efficiency
Solution Approach 1:
The patent applies preliminary action by constructing a memory demand sequence before actual memory allocation occurs. The system pre-calculates the execution sequence of operators in the network computation graph and determines memory demands in advance, allowing for optimized allocation that prevents fragmentation while maintaining high allocation speed.
Solution Approach 2:
The patent introduces dynamics by transitioning from static memory allocation to dynamic allocation based on operator execution sequences. The memory management system adapts to the actual computation flow, dynamically adjusting memory allocation strategies to match the runtime behavior of operators, thereby reducing fragmentation without sacrificing allocation efficiency.
2Device complexity
If memory is allocated without considering execution sequence, then allocation process is simpler, but memory fragmentation increases
Solution Approach 1:
The system performs preliminary analysis of the network computation graph to determine operator execution sequences and memory demands before actual allocation. This pre-computation simplifies the runtime allocation process while ensuring optimal memory utilization, as the allocation strategy is already tailored to the specific computation pattern.
Solution Approach 2:
The patent implements feedback mechanisms where the memory management system monitors and adjusts allocation based on the actual execution sequence of operators. By feeding back information about memory access patterns and operator dependencies, the system optimizes allocation strategies to minimize fragmentation while maintaining manageable complexity.
Data Source
AI summary
A memory management method, a storage medium, and an electronic device are provided. The method includes steps S21-S23. Step S21 includes obtaining, based on an execution sequence of operators in a network computation graph, a data stream sequence corresponding to the operators during operation, and constructing a memory demand sequence corresponding to the data stream sequence. Step S22 includes creating a target memory block in available memory spaces of a first memory. Step S23 includes allocating a memory space for each of memory demands in the memory demand sequence based on a sequencing number and a life cycle of each of the memory demands and a size of the target memory block. The presently disclosed method enhances memory utilization efficiency.