Memory Replication Service Reducing Network Traffic via Dirty Page Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The replication of memory data across hosts generates excessive network traffic, which can negatively impact performance, especially during live migration of virtual machines or when transferring large memory datasets, such as databases.
Innovation Solution
Implementing a replication service that utilizes checksums and dirty page tracking to determine which memory data is already available on the destination host, allowing it to be copied from local storage instead of being sent over the network, and selecting the destination host based on available resources and proximity to duplicate data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory data is replicated from source host to destination host over the network, then the destination host obtains the required memory data, but excessive network traffic is generated
Solution Approach 1:
The patent extracts only the necessary memory pages (dirty pages) that have been modified since the last replication point, rather than transferring all memory data. The hypervisor identifies and selects only those pages that contain changes, eliminating redundant data transfer over the network.
Solution Approach 2:
The system performs preliminary identification of dirty pages before initiating the replication transfer. The hypervisor tracks which memory pages have been modified and prepares a list of pages to be transferred in advance, allowing the replication process to start with a clear understanding of what data needs to be moved.
2Productivity
If all memory data is transferred over the network during live migration, then the virtual machine state is replicated, but network performance is degraded
Solution Approach 1:
The patent applies partial action by transferring only a subset of memory pages (those that are dirty/modified) rather than all pages. This selective approach reduces the total volume of data transferred over the network while still achieving complete virtual machine state replication, thereby improving network throughput during migration.
Solution Approach 2:
The memory space is segmented into individual pages, and the replication process operates at the page level rather than transferring the entire memory space. This segmentation allows the system to identify and transfer only the specific pages that contain changes, reducing overall network traffic during live migration.
3Reliability
If memory data replication is performed frequently, then data consistency is maintained, but network traffic increases
Solution Approach 1:
The hypervisor implements feedback by tracking which memory pages have been modified (dirty pages) and using this information to determine what data needs to be replicated. This feedback mechanism allows the system to maintain data consistency by replicating only the pages that have changed since the last replication point, rather than repeatedly transferring all memory data.
Solution Approach 2:
The system extracts only the modified memory pages from the complete memory space for replication. By identifying and selecting only the dirty pages that contain changes, the system maintains data consistency across replication events while minimizing the volume of data transferred over the network.
Data Source
AI summary
Technologies are described herein for reducing network traffic when replicating memory data across hosts. The memory data stored in a main memory of the host computer is replicated to a main memory of a second host computer. Memory data from the local data storage of the second host computer that is a duplicate of memory data from the main memory is identified. Instead of sending the memory data from the main memory that is duplicated, the duplicated memory is copied from the local storage to the main memory of the second host computer.