Gang Migration Deduplication for Data Center Network Overload

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Simultaneous live migration of multiple virtual machines (VMs) in data centers is network-intensive, causing adverse performance impacts due to high network traffic and resource overload, particularly in high-bandwidth low-delay environments, where existing methods fail to effectively reduce data transmission and migration time.

Innovation Solution

The implementation of gang migration with global deduplication (GMGD) identifies and tracks identical memory content across VMs, eliminating duplicate page transmissions through a distributed deduplication mechanism, using a combination of duplicate tracking and deduplication phases to reduce network traffic and migration time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If simultaneous live migration of multiple VMs is performed using traditional online compression methods, then migration can be completed, but network traffic is excessive and migration time is prolonged

Engineering Contradiction:

Improvemigration speedVSAvoidnetwork traffic volume

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent merges the migration streams of multiple VMs and applies cluster-wide deduplication to identify and eliminate duplicate memory pages across all migrating VMs. By combining the deduplication process at the cluster level rather than individually per VM, the system reduces total network traffic volume while maintaining migration productivity.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent performs a duplicate tracking phase before the actual migration to pre-identify redundant memory pages across all VMs in the cluster. This preliminary action allows the system to eliminate duplicates in advance, reducing the volume of data that needs to be transmitted during the migration execution phase.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If gang migration is performed to handle peak workloads and maintenance, then resource re-allocation is achieved, but core network links and switches are overloaded

Engineering Contradiction:

Improveresource re-allocation capabilityVSAvoidnetwork link overload

Core Design Contradiction:

Adaptability or versatilityVSStress or pressure

Solution Approach 1:

The patent combines multiple VM migration operations into a coordinated gang migration process with cluster-wide deduplication. By merging the deduplication function across the entire cluster, the system reduces the total network traffic generated by simultaneous migrations, thereby reducing stress on core network links and switches while maintaining resource re-allocation adaptability.

Inventive Principle:

Principle #5Merging (Combining)

3Productivity

If gang migration is performed, then multiple VMs can be migrated simultaneously, but performance at network edges deteriorates due to bandwidth competition

Engineering Contradiction:

Improvemigration throughputVSAvoidapplication performance degradation

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The patent performs duplicate tracking and identification before the actual migration data transfer. By pre-identifying redundant pages across all migrating VMs, the system reduces the volume of data that must traverse network edges during migration, thereby minimizing bandwidth competition and protecting application performance while maintaining migration throughput.

Inventive Principle:

Principle #10Preliminary action

4Quantity of substance

If deduplication is applied to reduce migration data, then network traffic is reduced, but coordination overhead increases

Engineering Contradiction:

Improvedata transmission volumeVSAvoidcoordination overhead

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent implements a cluster-wide deduplication mechanism that serves multiple migrating VMs simultaneously through a shared deduplication infrastructure. This universal approach allows the system to reduce data transmission volume for all VMs while amortizing the coordination overhead across the entire cluster, making the overhead manageable despite the increased scope.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS9372726B2Gang migration of virtual machines using cluster-wide deduplication

Publication Date: 2016.06.21 THE RES FOUNDATION FOR THE STATE UNIV OF NEW YORK

AI summary

Datacenter clusters often employ live virtual machine (VM) migration to efficiently utilize cluster-wide resources. Gang migration refers to the simultaneous live migration of multiple VMs from one set of physical machines to another in response to events such as load spikes and imminent failures. Gang migration generates a large volume of network traffic and can overload the core network links and switches in a data center. The present technology reduces the network overhead of gang migration using global deduplication (GMGD). GMGD identifies and eliminates the retransmission of duplicate memory pages among VMs running on multiple physical machines in the cluster. A prototype GMGD reduces the network traffic on core links by up to 51% and the total migration time of VMs by up to 39% when compared to the default migration technique in QEMU/KVM, with reduced adverse performance impact on network-bound applications.