Compute Kernel Halting for Shared Memory Coherency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Compute accelerator workloads are challenging to pause and resume due to their nature, leading to inefficiencies in resource utilization and migration between hosts, as they typically cannot be interrupted and restarted from the same point, causing compatibility issues and inefficiencies in load balancing and fault tolerance.
Innovation Solution
Implementing a method for fine-grain data coherency in shared memory regions, allowing for conditional halting points within compute kernels, enabling suspension and resumption of compute accelerator workloads, and translating between different data coherency models to facilitate migration across hosts with varying hardware platforms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compute accelerator workloads are executed without interruption, then computational efficiency is improved, but resource utilization and fault tolerance deteriorate due to inability to pause or migrate workloads
Solution Approach 1:
The compute kernel execution is segmented into multiple intervals with conditional halting points inserted between them. Each segment can be independently suspended and resumed, allowing workload migration while maintaining computational progress. The kernel execution flow is divided into: initialization phase, first compute kernel execution interval, suspension point, second compute kernel execution interval, and completion phase.
Solution Approach 2:
Data coherency is established in advance before workload migration by flushing write buffers and synchronizing shared memory regions between CPU and compute accelerator. This preliminary data coherency action ensures that when the workload is resumed on a different host, the data is already consistent and no additional synchronization delays are needed.
2Adaptability or versatility
If compute kernels are suspended frequently to enable migration, then workload adaptability is improved, but execution time increases due to repeated suspension and resumption overhead
Solution Approach 1:
Different data coherency models are applied to different memory regions based on their access patterns. Write buffers are flushed at specific halting points rather than continuously, and shared memory regions are synchronized only when needed for migration. This localized approach to data coherency minimizes the overhead of suspension operations while maintaining data consistency where required.
3Reliability
If data coherency is maintained across host migrations, then reliability is improved, but system complexity increases due to different hardware platforms and addressing modes
Solution Approach 1:
A virtualization layer acts as an intermediary between the compute kernel and the underlying hardware platform. This virtualization layer abstracts the differences between host computers, providing a unified interface for memory access and data coherency management. It handles the translation and synchronization of data between different hardware architectures, shielding the compute kernel from platform-specific complexities.
Solution Approach 2:
The system implements a universal data coherency mechanism that works across different hardware platforms and addressing modes (relative and absolute). The conditional halting points and buffer flushing mechanism are designed to be platform-agnostic, allowing the same compute kernel to be migrated between hosts with different architectures without modification.
Data Source
AI summary
The disclosure provides an approach for implementing fine grain data coherency of a memory region shared by an application within a virtual machine and a compute accelerator. The approach includes locating within a compute kernel a data write instruction to the shared memory region, and modifying the compute kernel to add a halting point after the data write instruction. The approach further includes configuring a virtualization system on which the virtual machine runs to set a value of a halt variable to true at an interval or in response to an occurrence of an event, wherein setting the halt variable to true causes the compute kernel to suspend execution at the conditional halting point.