Gang Migration Deduplication Reduces Network Overhead

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The simultaneous live migration of multiple virtual machines in a cluster results in significant network traffic overhead, which can overload datacenter networks and impact application performance, with existing technologies failing to effectively reduce this overhead while ensuring efficient migration and robustness.

Innovation Solution

The implementation of gang migration using global deduplication (GMGD), which identifies and tracks identical memory pages across multiple virtual machines within a cluster, suppressing the retransmission of duplicate pages and optimizing network traffic by transferring only unique pages, thereby reducing network load and migration time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If gang migration is performed to migrate multiple virtual machines simultaneously, then resource re-allocation and load balancing are improved, but network traffic overhead increases significantly

Engineering Contradiction:

Improvemigration efficiencyVSAvoidnetwork traffic

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent merges the migration traffic of multiple virtual machines by identifying and deduplicating identical memory pages across VMs. Instead of transmitting duplicate pages separately for each VM, the system combines them into a single transmission, thereby reducing overall network traffic while maintaining efficient simultaneous migration of multiple VMs.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent performs preliminary deduplication of memory pages before migration by computing hash values and identifying duplicate pages across multiple VMs. This preliminary action allows the system to prepare a deduplicated set of pages to be transmitted, reducing the network traffic that would otherwise be generated during the actual migration process.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If gang migration transfers hundreds of Gigabytes of data over the network, then complete VM state migration is achieved, but core links and switches become overloaded

Engineering Contradiction:

Improvemigration completenessVSAvoidnetwork load

Core Design Contradiction:

ReliabilityVSStress or pressure

Solution Approach 1:

The patent merges identical memory pages from multiple VMs into a single deduplicated set, significantly reducing the volume of data that must traverse core network links. This merging approach ensures complete VM state migration while preventing network overload by eliminating redundant data transmissions across the network infrastructure.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent extracts and removes duplicate memory pages from the migration data set before transmission. By taking out redundant pages that are identical across multiple VMs, the system reduces the total data volume requiring network transmission, thereby preventing core links and switches from becoming overloaded while still achieving complete migration of unique VM states.

Inventive Principle:

Principle #2Taking out (Extraction)

3Productivity

If migration traffic competes with application bandwidth requirements at network edges, then gang migration is performed, but application performance degrades

Engineering Contradiction:

Improvemigration throughputVSAvoidapplication performance impact

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The patent merges duplicate memory pages across VMs to reduce overall migration traffic volume. This reduction in traffic volume decreases the competition between migration traffic and application traffic at network edges, thereby minimizing performance degradation of applications while still achieving efficient gang migration throughput.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent performs preliminary deduplication of migration data before transmission, reducing the amount of traffic that will compete with application bandwidth at network edges. This preliminary action ensures that less migration traffic needs to share the network infrastructure with applications, thereby reducing performance impact while maintaining migration throughput.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10156986B2Gang migration of virtual machines using cluster-wide deduplication

Publication Date: 2018.12.18 THE RES FOUNDATION FOR THE STATE UNIV OF NEW YORK

AI summary

Gang migration refers to the simultaneous live migration of multiple Virtual Machines (VMs) from one set of physical machines to another in response to events such as load spikes and imminent failures. Gang migration generates a large volume of network traffic and can overload the core network links and switches in a datacenter. In this paper, we present an approach to reduce the network overhead of gang migration using global deduplication (GMGD). GMGD identifies and eliminates the retransmission of duplicate memory pages among VMs running on multiple physical machines in the cluster. The design, implementation and evaluation of a GMGD prototype is described using QEMU/KVM VMs. Evaluations on a 30-node Gigabit Ethernet cluster having 10GigE core links shows that GMGD can reduce the network traffic on core links by up to 65% and the total migration time of VMs by up to 42% when compared to the default migration technique in QEMU/KVM. Furthermore, GMGD has a smaller adverse performance impact on network-bound applications.