RDMA-Based Dirty Data Recovery for High Availability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing high availability (HA) approaches in computing environments fail to recover dirty data from a source host system in case of an unrecoverable OS/hypervisor kernel error, as they require the OS/hypervisor to be operational for data synchronization.
Innovation Solution
The implementation of Remote Direct Memory Access (RDMA) technology allows for data transfer between host systems without involving the OS/hypervisor, enabling the failover host system to recover dirty data from the source host system even in the event of an unrecoverable kernel error by establishing an RDMA connection and using RDMA-based recovery agents to synchronize and reconstruct the data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If OS/hypervisor-based data synchronization is used, then data can be synchronized under normal operating conditions, but data recovery fails when OS/hypervisor kernel errors occur
Solution Approach 1:
The patent introduces RDMA (Remote Direct Memory Access) technology as an intermediary mechanism that enables direct memory-to-memory data transfer between source and failover host systems, bypassing the OS/hypervisor layer. This intermediary approach allows dirty data to be synchronized even when the OS/hypervisor is non-operational due to kernel errors, thereby resolving the contradiction between maintaining reliability under normal conditions and adapting to severe failure modes.
2Reliability
If periodic flushing to shared storage backend is implemented, then data durability is improved, but recovery time increases due to OS involvement
Solution Approach 1:
The patent extracts the data synchronization function from the OS/hypervisor layer and implements it directly at the hardware/memory level using RDMA technology. By taking out the OS involvement from the critical data transfer path, the system achieves both data durability through periodic flushing and reduced recovery time, as the RDMA-based synchronization can proceed independently of OS state after kernel errors.
3Device complexity
If OS/hypervisor operational state is assumed, then system complexity is reduced, but system robustness deteriorates under severe failures
Solution Approach 1:
The patent segments the data synchronization function into two independent pathways: the traditional OS/hypervisor-based pathway for normal operations, and the RDMA-based direct memory access pathway for failure recovery. This segmentation allows the system to maintain simplicity under normal conditions while gaining robustness under severe failures, as the RDMA pathway operates independently of OS/hypervisor state and can recover data even when the OS is non-operational.
Data Source
AI summary
Techniques for implementing RDMA-based recovery of dirty data in remote memory are provided. In one set of embodiments, upon occurrence of a failure at a first (i.e., source) host system, a second (i.e., failover) host system can allocate a new memory region corresponding to a memory region of the source host system and retrieve a baseline copy of the memory region from a storage backend shared by the source and failover host systems. The failover host system can further populate the new memory region with the baseline copy and retrieve one or more dirty page lists for the memory region from the source host system via RDMA, where the one or more dirty page lists identify memory pages in the memory region that include data updates not present in the baseline copy. For each memory page identified in the one or more dirty page lists, the failover host system can then copy the content of that memory page from the memory region of the source host system to the new memory region via RDMA.