RDMA Management During Virtual Machine Migration

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In virtualized computing environments, the migration of virtual machines disrupts remote direct memory access (RDMA) communications, leading to interruptions and potential data loss due to changes in connectivity and transport availability between virtual machines.

Innovation Solution

A method is introduced to manage RDMA operations by suspending and resuming operations during virtual machine migration, using a first channel for initial suspension and establishing a second channel post-migration to ensure seamless RDMA communication, with options to use RDMA, TCP, or memory copying based on host capabilities.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If virtual machine migration is performed to enable flexible resource allocation, then adaptability and resource utilization are improved, but RDMA communication continuity deteriorates causing interruptions and potential data loss

Engineering Contradiction:

Improveresource allocation flexibilityVSAvoidRDMA communication continuity

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system performs preliminary actions by suspending RDMA operations before migration and coordinating with peer VMs to suspend their RDMA operations targeting the migrating VM. This preliminary suspension prevents data loss and communication interruptions during migration, allowing the VM to be moved while maintaining RDMA reliability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The hypervisor acts as an intermediary between the virtualized RDMA stack and physical RDMA resources. It manages the suspension and resumption of RDMA operations, coordinates with peer VMs through control channels, and handles the reestablishment of RDMA connections after migration, thereby resolving the contradiction between migration flexibility and communication continuity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If RDMA operations are suspended during migration, then data loss is prevented, but communication latency increases due to operation interruption and reestablishment

Engineering Contradiction:

Improvedata loss preventionVSAvoidcommunication latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system maintains continuity of useful action by quickly resuming RDMA operations after migration completion. The hypervisor reestablishes RDMA connections and resumes operations on the second host, minimizing the interruption period. This approach prevents data loss while reducing communication latency by ensuring rapid resumption of RDMA operations.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system rushes through the migration process by performing the suspension, migration, and reestablishment operations in rapid succession. The coordinated suspension of RDMA operations on both local and peer VMs allows the migration to complete quickly, minimizing the time loss while ensuring data integrity.

Inventive Principle:

Principle #21Skipping (Rushing through)

3Adaptability or versatility

If migration to hosts without native RDMA capabilities is allowed, then adaptability is improved, but RDMA performance deteriorates due to lack of hardware acceleration

Engineering Contradiction:

Improvehost compatibilityVSAvoidRDMA communication performance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system uses memory copying as a fallback mechanism when migrating to hosts without native RDMA capabilities. The virtualized RDMA stack implements memory copying to simulate RDMA operations, allowing the VM to maintain RDMA communication functionality on hosts with different hardware capabilities, though with reduced performance compared to native RDMA hardware acceleration.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system changes the transport parameter from native RDMA hardware acceleration to alternative methods (TCP or memory copying) when migrating to hosts without RDMA capabilities. This parameter change allows the system to adapt to different host environments while maintaining RDMA communication functionality, trading performance for adaptability.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10417174B2Remote direct memory access in a virtualized computing environment

Publication Date: 2019.09.17 VMWARE INC

AI summary

A method of managing remote direct memory access (RDMA) to a virtual computing instance includes suspending locally initiated RDMA operations of the virtual computing instance executing on a first host prior to a migration of the virtual computing instance to a second host. The first host includes a first hypervisor and the second host includes a second hypervisor. The method further includes requesting a peer to suspend remotely initiated RDMA operations that target the virtual computing instance through a first channel, establishing after the migration, a second channel between the peer and the second hypervisor that supports execution of the virtual computing instance on the second host, configuring a virtual object of the second hypervisor on the second host to use the second channel for the locally initiated RDMA operations, and requesting the peer to resume the remotely initiated RDMA operations using the second channel.