Distributed Array Memory for Heterogeneous Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing programming models for heterogeneous computing systems face complexities in managing multiple memory spaces, particularly in ensuring data coherence and optimizing memory access across different levels of the memory hierarchy, leading to inefficient programming and compiler optimizations.
Innovation Solution
The implementation of a distributed array memory structure, which abstracts multiple memory spaces by declaring a global set of local memory regions, allowing segments to be allocated and bound to physical memory regions, and enabling persistent or shared access within workgroups, thereby simplifying memory management and optimizing data access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a distributed array memory structure is implemented to abstract multiple memory spaces, then memory management overhead is reduced and portability is improved, but the complexity of managing data coherence and access across different memory levels increases
Solution Approach 1:
The memory space is segmented into distributed array structures where each element can be independently allocated to different memory levels (L1 cache, L2 cache, main memory). This segmentation allows the system to manage complex multi-level memory hierarchies through a simplified uniform access interface, reducing memory management overhead while maintaining adaptability for data coherence through localized segment control.
Solution Approach 2:
The distributed array structure acts as an intermediary layer between the programmer and the physical memory hierarchy. It provides a uniform access interface that abstracts away the complexity of managing data coherence across L1 cache, L2 cache, and main memory, while still enabling optimized access patterns and persistent storage capabilities.
2Adaptability or versatility
If segments are allocated at run time to match executing groups, then memory allocation flexibility is improved, but the overhead of runtime allocation and binding increases
Solution Approach 1:
The distributed array structure is declared and initialized with allocation parameters before program execution begins. Segment allocation patterns are pre-configured based on expected workgroup sizes and memory requirements, allowing the runtime system to efficiently bind segments to physical memory regions without performing complex allocation calculations during execution, thus reducing runtime overhead while maintaining flexibility.
3Extent of automation
If persistent distributed arrays are used to enable data persistence across dispatches, then automated data persistence is improved, but memory retention requirements and resource management complexity increase
Solution Approach 1:
The distributed array structure is designed to support multiple persistence modes (persistent, non-persistent, shared, private) within a single unified framework. This universality allows the same data structure to automatically adapt to different persistence requirements without requiring separate management mechanisms, thereby automating data persistence while keeping resource management complexity manageable through standardized interfaces.
Data Source
AI summary
In a computing system, memory may be managed by using a distributed array, which is a global set of local memory regions. A segment in the distributed array is allocated and is bound to a physical memory region. The segment is used by a workgroup in a dispatched data parallel kernel, wherein a workgroup includes one or more work items. When the distributed array is declared, parameters of the distributed array may be defined. The parameters may include an indication whether the distributed array is persistent (data written to the distributed array during one parallel dispatch is accessible by work items in a subsequent dispatch) or an indication whether the distributed array is shared (nested kernels may access the distributed array). The segment may be deallocated after it has been used.