Splitter-Journal Kernel Replication for Near-Zero IO Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Any-point in time (any-PIT) OS kernel-based replication solutions incur significant Input/Output (IO) latency due to the kernel module intercepting and synchronously sending IOs to a replication appliance, which is not keeping pace with memory and storage write speeds, leading to performance issues with high-performance applications.
Innovation Solution
Employing a splitter-journal maintained in-memory and sent asynchronously to a replication appliance, along with the use of non-volatile memory (NVM) to reduce IO latency while preserving any-PIT restore capabilities, by intercepting IOs, writing them to the journal, and forwarding them to storage without adding latency to the IO path.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the kernel module intercepts and synchronously sends IOs to the replication appliance, then replication accuracy is maintained, but IO latency increases significantly
Solution Approach 1:
The patent segments the replication process into two independent parts: (1) the primary IO path from application to storage, and (2) the replication path from storage to replication appliance. By using storage-level replication instead of kernel-level interception, the system divides the IO flow so that application IOs are not blocked by replication operations, eliminating the latency while maintaining replication accuracy through storage-ensured data integrity.
Solution Approach 2:
The patent introduces storage as an intermediary layer between the application and the replication appliance. Instead of the kernel module directly intercepting and forwarding IOs, the storage system acts as a mediator that receives IOs from the application and independently handles replication to the appliance. This intermediary approach allows the replication process to occur without interfering with the application's IO performance.
2Stability of the object's composition
If synchronous IO transmission to replication appliance is used, then data consistency is ensured, but communication speed becomes the bottleneck
Solution Approach 1:
The patent implements preliminary action by ensuring data is fully written to and confirmed in the storage system before triggering replication. The storage appliance prepares the data for replication only after the primary IO is complete, which maintains data consistency without requiring synchronous communication between the kernel module and replication appliance during the IO operation.
Solution Approach 2:
The patent enables continuous IO operations at the application level while replication occurs continuously in the background at the storage level. The useful action of data protection continues without interruption to the application's IO performance, as the storage system handles replication asynchronously after receiving confirmed data from the application.
Data Source
AI summary
One example method includes intercepting an IO issued by an application, writing the IO and IO metadata to a splitter journal in NVM, forwarding the IO to storage, and asynchronous with operations occurring along an IO path between the application and storage, evacuating the splitter journal by sending the IO and IO metadata from the splitter journal to a replication site. In this example, sending the IO and IO metadata from the journal to the replication site does not increase a latency associated with the operations on the IO path.