NAS Journaling for Low-Latency Data Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication solutions for shared memory access are unreliable and inefficient, often requiring compatible vendor services, introducing latency, and suffering from single-point-of-failure issues and vendor lock-in.
Innovation Solution
A network attached storage (NAS) memory controller and NAS client configuration that allows for direct, low-latency communication, eliminating the need for gateways and programmatic access to the NAS source, while synchronizing file operations and resolving conflicts to ensure reliable data replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If gateway-based data replication solutions are used, then data can be replicated between NAS devices, but latency is introduced and single-point-of-failure problems occur
Solution Approach 1:
The patent removes the gateway component from the data replication architecture. Instead of routing data through a gateway, the system establishes direct peer-to-peer communication channels between NAS devices, eliminating the intermediary that causes latency and single-point-of-failure issues while maintaining data replication functionality
Solution Approach 2:
The patent implements journal copying where the source NAS device creates a journal of data changes and transmits it to the target NAS device. This copying mechanism allows efficient data replication without requiring continuous synchronization or gateway mediation, reducing both latency and dependency on intermediate components
2Reliability
If conventional data replication solutions with agents are used, then data can be captured and replicated, but race conditions occur when data is reread and overwritten
Solution Approach 1:
The patent implements preliminary journaling where data changes are recorded in a journal before being applied to the file system. This preliminary action creates a consistent snapshot of changes that can be reliably replicated without rereading data, preventing race conditions while simplifying the replication mechanism
Solution Approach 2:
The patent establishes continuous journal transmission from source to target NAS device, maintaining an ongoing replication stream that eliminates the need for periodic rereading of data. This continuous action ensures data consistency without introducing race conditions or complex synchronization mechanisms
3Productivity
If snapshot API-based replication is used, then file system changes can be detected, but entire files must be replicated even when only small portions change
Solution Approach 1:
The patent implements journal-based replication that tracks and transmits only the specific data blocks that have changed, rather than replicating entire files. This local quality approach sends precisely the minimal necessary data from source to target, improving efficiency and reducing bandwidth waste
Solution Approach 2:
The patent changes the replication parameter from file-level snapshots to block-level journaling. By tracking changes at the data block level rather than file level, the system can detect and replicate only the specific portions of files that have changed, dramatically improving productivity and reducing bandwidth consumption
4Ease of manufacture
If vendor-specific replication solutions are used, then data replication can be implemented, but vendor lock-in occurs
Solution Approach 1:
The patent implements a universal journaling protocol that enables data replication between NAS devices from different vendors. The journal format and transmission mechanism are vendor-agnostic, allowing the source NAS device to replicate data to target devices from any manufacturer, eliminating vendor lock-in while maintaining ease of implementation
Data Source
AI summary
A NAS memory controller is provided that is operatively connected to one or more NAS clients for shared memory access, where each NAS client includes a clock and is operatively connected to other NAS clients of the one or more NAS clients through a low-latency communications network. The NAS memory controller is configured to transmit an indication of a stop time and a restart time to each of the one or more NAS clients, and receive a journal of file operations from each of the one or more NAS clients after the stop time arrives. The NAS memory controller is further configured to determine a status of each journal.