Compute Accelerator Workload Migration via Kernel State Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Compute accelerators (CAs) cannot pause and resume workloads from the same execution point, leading to inefficiencies in resource utilization and fault tolerance, as they must restart from the beginning if interrupted, which is problematic for shared resources and migration between hosts.
Innovation Solution
Implementing a method that uses a custom translation lookaside buffer (TLB) for shared virtual memory between a CPU and CAs, allowing the kernel to access a custom TLB and suspend/resume workloads, enabling conditional halting points and virtual shared memory for workload migration and fault tolerance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compute accelerators execute workloads continuously without pause capability, then execution speed and productivity are improved, but fault tolerance and resource sharing flexibility deteriorate
Solution Approach 1:
The system performs preliminary actions by capturing the complete execution state (registers, memory pointers, program counter, TLB entries) at the moment of suspension. This state information is stored and later used to resume execution from the exact same point, enabling fault tolerance without losing productivity
Solution Approach 2:
The system introduces dynamic pause/resume capability to the previously static continuous execution model. The compute accelerator can now dynamically transition between executing and suspended states, allowing fault tolerance while maintaining high productivity through rapid resumption from captured state
2Ease of operation
If compute accelerators cannot pause workloads, then execution simplicity is maintained, but resource sharing and load balancing capability deteriorate
Solution Approach 1:
The system introduces a state capture mechanism as an intermediary between the compute accelerator execution and the resource management system. This intermediary captures execution state information and enables pause/resume functionality, allowing resource sharing and load balancing without significantly complicating the execution process
Solution Approach 2:
The system creates a copy of the execution state (registers, memory pointers, program counter, TLB entries) at suspension time. This copied state information allows the workload to be resumed later from the same point, enabling resource sharing flexibility while maintaining execution simplicity through direct state restoration
3Device complexity
If compute accelerators restart workloads from beginning upon interruption, then implementation simplicity is maintained, but time loss and productivity deterioration
Solution Approach 1:
The system performs preliminary capture of execution state information (registers, memory pointers, program counter, TLB entries) at the moment of interruption. This preliminary action enables rapid resumption from the exact point of interruption, dramatically reducing time loss while adding minimal implementation complexity
Solution Approach 2:
The system recovers the execution state information that would otherwise be lost upon interruption. By capturing and storing the complete state (registers, memory pointers, program counter, TLB entries), the system can discard the need to restart from beginning and recover to the exact interruption point, eliminating time loss with minimal complexity increase
4Adaptability or versatility
If custom TLB is implemented for virtual memory translation, then migration capability is improved, but device complexity and memory management overhead increase
Solution Approach 1:
The custom TLB structure serves multiple functions: virtual memory translation, state capture during suspension, and migration support. By making the TLB multi-functional, the system achieves workload migration capability without proportionally increasing device complexity or memory management overhead
Solution Approach 2:
The system modifies the TLB parameters and structure to include virtual address space mappings between host and guest. This parameter change enables the TLB to support workload migration while maintaining efficient memory management, achieving versatility without excessive complexity increase
Data Source
AI summary
The disclosure provides an approach for modifying a kernel by adding conditional halting points. The disclosure also provides an approach for modifying a kernel by implementing a virtual shared memory between an application running on a CPU and a workload running on a compute accelerator. The disclosure provides an approach for setting up the kernel and its working set on a compute accelerator, executing the workload, suspending the workload, and then resuming the workload at a later time, optionally on a different host computer.