VM Memory Copy During Migration Using Loose Trace Installation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The migration of virtual machines in virtualized computing systems often results in a significant drop in performance due to the memory copy phase, especially for large VMs with hundreds of virtual CPUs and terabytes of memory, as existing methods like stop-based trace installation halt all virtual CPUs during write trace installation, disrupting guest software execution.
Innovation Solution
The implementation of 'loose trace' installation, where the VM migration software uses less than all virtual CPUs for write trace installation, allowing guest software to continue executing and implementing lazy TLB flushing, reducing the impact on performance by handling races between trace installation and tracing fires, and using large page tracing to minimize disruptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If stop-based trace installation is used to install write traces during VM migration, then memory copy accuracy is ensured, but guest software performance deteriorates significantly due to all virtual CPUs being halted
Solution Approach 1:
The patent divides the virtual CPU resources into two segments: one set of virtual CPUs is dedicated to installing write traces, while another set continues to execute guest software. This segmentation allows the trace installation process to proceed without completely halting guest operations, thereby maintaining both memory copy accuracy and guest software performance to some extent.
Solution Approach 2:
The patent implements a dynamic approach where the system transitions from a static stop-based model to a dynamic loose trace installation model. The hypervisor dynamically manages the installation of write traces on memory pages while allowing guest software to continue executing, adapting the trace installation process to coexist with ongoing computational tasks rather than requiring a complete stop.
2Manufacturing precision
If all virtual CPUs are used for write trace installation during memory copy, then trace installation completeness is improved, but migration time increases due to VM quiescing
Solution Approach 1:
The patent employs preliminary action by installing write traces on memory pages in advance of the actual migration switch-over point. The loose trace installation allows traces to be set up progressively while the VM continues running, so that by the time migration is initiated, the trace installation is already complete or nearly complete, eliminating the need for a time-consuming quiesce period.
Solution Approach 2:
The patent maintains continuity of useful action by allowing guest software to continue executing on some virtual CPUs throughout the trace installation process. This continuous execution ensures that the VM remains productive during migration preparation, and the trace installation proceeds without interruption, completing the process more efficiently than stop-based methods.
3Productivity
If loose trace installation is used allowing parallel execution, then guest software performance is maintained, but handling races between trace installation and tracing fires becomes more complex
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of a hypervisor-managed trace installation system that mediates between the trace installation process and guest software execution. The hypervisor acts as an intermediary layer that coordinates trace installation on memory pages while managing trace fires, preventing direct conflicts and race conditions between these processes through controlled interaction protocols.
Solution Approach 2:
The patent implements feedback mechanisms where the system monitors trace installation status and trace fire events in real-time. When a trace fire occurs during loose trace installation, the system receives feedback about the race condition and adjusts its behavior accordingly, such as pausing trace installation temporarily or handling the trace fire event appropriately, thereby managing the complexity of concurrent operations through active monitoring and response.
Data Source
AI summary
A virtual machine (VM) is migrated from a source host to a destination host in a virtualized computing system, the VM having a plurality of virtual central processing units (CPUs). The method includes copying, by VM migration software executing in the source host and the destination host, memory of the VM from the source host to the destination host by installing, at the source host, write traces spanning all of the memory and then copying the memory from the source host to the destination host over a plurality of iterations; and performing switch-over, by the VM migration software, to quiesce the VM in the source host and resume the VM in the destination host. The VM migration software installs write traces using less than all of the virtual CPUs, and using trace granularity larger than a smallest page granularity.