Multi-Accelerator Offload Selection by Core-to-Accelerator Cost
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems fail to efficiently utilize on-chip and off-chip accelerators due to varying core-to-accelerator access costs and non-deterministic latency, leading to inefficient resource utilization and performance issues in multi-socket platforms.
Innovation Solution
Implement a method to dynamically select accelerators based on core-to-accelerator cost metrics, using ACPI facilities to identify and utilize the closest available accelerator for offloaded workloads, ensuring reduced latency and improved deterministic performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If applications move around cores among different dies and sockets, then core resource utilization improves through statistical multiplexing, but data movement path efficiency deteriorates due to ineffective interconnect resource utilization
Solution Approach 1:
The system dynamically tracks application thread locations across cores and automatically updates accelerator assignments based on current core positions. This dynamic adaptation allows the system to maintain optimal core-to-accelerator mappings even as threads migrate, resolving the contradiction between resource utilization and latency by continuously optimizing the data movement path.
Solution Approach 2:
The system implements feedback mechanisms that monitor application thread locations and interconnect latency metrics in real-time. Based on this feedback, the accelerator management software recalculates optimal accelerator assignments and updates the mappings accordingly, ensuring that data movement paths remain efficient while allowing flexible core resource utilization.
2Device complexity
If uniform accelerator access cost is assumed, then system complexity is reduced, but actual performance deteriorates due to varying core-to-accelerator latency
Solution Approach 1:
The system changes the parameter representation from a single uniform access cost to a multi-dimensional cost model that includes core-to-accelerator latency, interconnect distance, and accelerator availability. This parameter transformation enables the system to account for varying access costs without excessive complexity by using standardized metrics that can be computed and stored efficiently.
Solution Approach 2:
The system pre-calculates and stores core-to-accelerator cost metrics in lookup tables before application execution. This preliminary action allows the runtime system to quickly query optimal accelerator assignments without performing complex calculations during thread migration, thus maintaining low system complexity while achieving high performance through pre-computed optimal paths.
3Reliability
If static resource allocation is used for core affinity, then deterministic performance is improved, but resource waste increases due to inability to share cores
Solution Approach 1:
The system dynamically adjusts accelerator assignments based on real-time core occupancy and application requirements. When cores become available through completion of previous tasks, the system reassigns accelerators to new applications on those cores, maintaining deterministic performance characteristics while enabling statistical multiplexing to reduce resource waste.
Solution Approach 2:
The accelerator management system serves multiple functions: it provides deterministic performance guarantees, enables statistical multiplexing of core resources, tracks thread locations across migrations, and optimizes data movement paths. This multi-functionality resolves the contradiction by integrating performance determinism with resource efficiency in a unified management framework.
Data Source
AI summary
Methods, apparatus, and software for efficient accelerator offload in multi-accelerator frameworks. One multi-accelerator framework employs a compute platform including a plurality of processor cores and a plurality of accelerator devices. An application is executed on a first core and a portion of the application workload is offloaded to a first accelerator device. In connection with moving execution of the application to a second core, a second accelerator devices to be used for the offloaded workload is selected based on core-to-accelerator cost information for the second core. This core-to-accelerator cost information includes core-accelerator cost information for combinations of core-accelerator pairs, which are based, at least on part, on latencies projected for interconnect paths between cores and accelerators. Both single-socket and multi-socket platform are supported. The solutions include mechanisms for moving offloaded workloads for multiple accelerator devices, as well as synchronizing accelerator operations and workflows.