Cache Coherent FPGA Post-Copy VM Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In live virtual machine migration, post-copy migration encounters inefficiencies due to page faults at the destination host, which are costly and time-consuming, especially for large pages.
Innovation Solution
The method involves indicating to the destination host that all virtual machine pages are present, using RDMA to handle cache misses by obtaining and sending cache lines from the source host to the destination host, thereby avoiding page faults and efficiently copying pages during migration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If post-copy migration is used to start the virtual machine on the destination host before pages are transferred, then the migration can proceed without waiting for complete page transfer, but the system encounters many page faults that are expensive and time-consuming to service
Solution Approach 1:
The system pre-loads pages into the destination host's memory before the virtual machine needs to access them. When a page fault is predicted or detected, the missing page is fetched from the source host and loaded into the destination memory in advance, so that when the virtual machine actually needs the page, it is already available and no time is lost servicing the fault.
Solution Approach 2:
A page fault handling mechanism acts as an intermediary between the virtual machine and the page transfer process. This intermediary intercepts page fault requests, manages the fetching of missing pages from the source host, and delivers them to the destination memory, thereby decoupling the virtual machine execution from the page transfer timing.
2Quantity of substance
If large pages (2 MB to 1 GB) are transferred during migration, then the memory capacity is sufficient, but the time to obtain a page to satisfy a page fault becomes excessive
Solution Approach 1:
Large pages are pre-fetched and loaded into the destination host's memory before they are needed by the virtual machine. The system anticipates which large pages will be needed and transfers them in advance, so that when the virtual machine requires a large page, it is already resident in memory and can be accessed immediately without causing a time-consuming page fault.
Solution Approach 2:
The migration process is segmented into multiple parallel operations: page transfer, page fault handling, and virtual machine execution. While large pages are being transferred in the background, the virtual machine can continue execution with pages already loaded, and page faults are handled as separate, non-blocking operations that do not stall the overall migration process.
Data Source
AI summary
Disclosed are embodiments for migrating a virtual machine (VM) from a source host to a destination host while the virtual machine is running on the destination host. The system includes an RDMA facility connected between the source and destination hosts and a device coupled to a local memory, the local memory being responsible for memory pages of the VM instead of the source host. The device is configured to copy pages of the VM to the destination host and to maintain correct operation of the VM by monitoring coherence events, such as a cache miss, caused by the virtual machine running on the destination host. The device services these cache misses using the RDMA facility and copies the cache line satisfying the cache miss to the CPU running the VM. The device also tracks the cache misses to create an access pattern that it uses to predict future cache misses.