Pooled Circular Buffers for Spatial Compute Memory Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spatial compute architectures have limited local memory, which can impact runtime performance due to the significant data consumption and production requirements of applications, and existing memory management techniques lead to inefficient use of memory resources.
Innovation Solution
Implementing dynamically pooled allocations of memory buffers, specifically using a pooled synchronized circular buffer, which reduces memory footprint by combining multiple buffers into a shared memory pool based on access patterns and execution times of producer and consumer processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate synchronized circular buffers are allocated for each producer-consumer pair, then synchronization between producers and consumers is ensured, but memory footprint increases significantly
Solution Approach 1:
Multiple separate synchronized circular buffers allocated for different producer-consumer pairs are merged into a single pooled synchronized circular buffer. The pool manager dynamically allocates and deallocates buffer segments within the pooled buffer for different producer-consumer pairs, reducing total memory footprint while maintaining synchronization guarantees through enforced access sequences and ownership tracking.
2Device complexity
If static memory allocation is used for local data memory, then memory management is simplified, but runtime performance degrades due to insufficient memory for significant data consumption and production
Solution Approach 1:
The system transitions from static memory allocation to dynamic memory allocation through a pool manager that operates at runtime. The pool manager dynamically allocates buffer segments to producer-consumer pairs based on actual memory needs and usage patterns, enabling the system to adapt to varying data consumption and production requirements while maintaining manageable complexity through automated allocation policies.
3Reliability
If multiple separate buffers are used for different producer-consumer pairs, then each pair has dedicated memory resources, but memory utilization efficiency decreases
Solution Approach 1:
The pooled buffer system enables dynamic recovery and reuse of memory segments. When a producer-consumer pair finishes using a buffer segment, the pool manager recovers the segment and makes it available for allocation to other producer-consumer pairs. This continuous cycle of allocation, usage, recovery, and reallocation ensures high memory utilization efficiency while maintaining dedicated resource guarantees through controlled access sequences.
Data Source
AI summary
Dynamically pooled allocation of memory buffer on spatial compute architectures, including analyzing, at compile-time, access patterns (e.g., cyclo-static execution/firing rules) of consumer and/or producer processes that have shared access to local memory of one or more compute tiles, and identifying situations in which multiple buffers can be replaced with a pooled buffer having a memory footprint that is less than a sum of the memory footprints of the multiple buffers. A compiler may identify instances of mutual exclusiveness in the execution patterns of the processes, differences in execution times between compute kernels of the processes, and/or variations in execution times of the kernels. The compiler may generate controller code and/or configuration parameters to enforce memory allocation/mapping at application run-time.