Runtime Memory Transfer Routing for Heterogeneous Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Heterogeneous computing systems face inefficiencies in data transfer due to unknown task allocations at runtime, leading to underutilization of high-performance data transfer facilities and suboptimal system performance.
Innovation Solution
Maintaining metadata in the form of validity flags to identify and utilize the most efficient source memory for data transfers based on runtime task allocations, enabling direct high-performance data transfers between processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-programmed memory transfers using hub-and-spoke architecture are used, then data transfer can be performed without knowing task allocations, but high-performance direct data transfer facilities are left unutilized
Solution Approach 1:
The system dynamically determines data transfer commands at runtime based on actual task allocations to processors. The memory manager queries task scheduler to obtain runtime information about which processors are executing which tasks, then selects the most efficient data transfer path accordingly. This dynamic adaptation enables utilization of high-performance direct data transfer facilities when available, while maintaining flexibility to use alternative paths when needed.
2Device complexity
If manually coded data transfers are used, then overhead is reduced somewhat, but task allocations not being known until runtime still prevents utilization of high-performance data transfer facilities
Solution Approach 1:
The invention introduces a memory manager as an intermediary component that acts as a bridge between the task scheduler and the data transfer mechanism. The memory manager queries the task scheduler to obtain runtime task allocation information, then uses this information to determine the optimal data transfer command. This intermediary layer enables automated selection of high-performance data transfer paths without requiring manual coding, while still achieving the performance benefits of runtime-aware transfer optimization.
3Productivity
If processors are constrained to specific task assignments, then high-performance data transfer facilities can be utilized, but task scheduler's ability to optimize system performance is limited
Solution Approach 1:
The system maintains full task scheduling flexibility by determining data transfer commands dynamically at runtime based on actual task allocations. The memory manager queries the task scheduler to obtain runtime information about processor-task mappings, then selects the most efficient data transfer path accordingly. This dynamic approach allows the task scheduler to freely optimize task assignments without pre-constraining processor selections, while still enabling utilization of high-performance direct data transfer facilities when the runtime allocation supports it.
Data Source
AI summary
In a heterogeneous computing environment, a validity flag indicates whether a given memory has a valid copy of a given data object. When the data object is required at a target memory for a scheduled task, another (source) memory having a valid copy of the data object is identified. A protocol guides selection of the source memory based on efficiency of data transfer to the target memory. A data transfer command is generated at runtime, when tasks have been scheduled among respective processors. Execution of the command copies the valid data object from the source memory to the target memory. Superior performance is achieved even in extremely heterogeneous environments. Performance results, variations, and applications are presented.