Multi-site Clustered Data Replication for Disaster Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cluster architectures for data backup and recovery may lead to significant data loss when a portion of the data is unavailable due to component failures, which is not tolerable for businesses relying on high data availability and disaster recovery.
Innovation Solution
A multi-site clustered data intake and query system is implemented, where data is replicated across geographically dispersed sites, using a master device to manage generation identifiers (GEN_IDs) and designate primary and secondary indexers for data storage and retrieval, ensuring data availability and minimizing downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is spread across a large number of cluster members, then the likelihood of losing large amounts of data is minimized, but loss of even a portion of the data may be significant and not tolerable
Solution Approach 1:
The patent segments data into discrete units called 'data slices' that can be independently replicated and distributed across cluster members. Each data slice is assigned to a primary member and copied to secondary members, allowing the system to maintain data availability while controlling the granularity of data distribution and recovery.
Solution Approach 2:
The patent implements local quality by maintaining different roles (primary vs. secondary) for different cluster members regarding the same data slice. The primary member holds the authoritative copy while secondary members hold replicas, allowing the system to optimize both data availability and recovery capabilities at different locations within the cluster.
2Reliability
If data is replicated across multiple cluster components, then data availability is improved, but device complexity increases
Solution Approach 1:
The patent creates a universal data slice structure that can be replicated across any cluster member regardless of its specific function or location. The same primary-secondary relationship model applies to all data slices, simplifying the overall architecture while enabling flexible replication across heterogeneous cluster components.
Solution Approach 2:
The patent uses simple copying mechanisms to replicate data slices from primary to secondary members. Instead of complex synchronization protocols, the system creates replicated copies that can be independently managed, reducing the complexity of data replication while maintaining availability and enabling straightforward recovery processes.
Data Source
AI summary
A method of data replication in a clustered computing environment comprises receiving, at a selected indexer within a plurality of indexers in a cluster, data from a forwarder indexer, wherein the selected indexer is designated as a primary indexer for the data, wherein the primary indexer has primary responsibility for responding to search queries pertaining to the data, wherein the cluster comprises a plurality of sites. The method further comprises receiving, at the selected indexer, data replication instructions, wherein the data replication instructions comprise a number of other indexers in the cluster for storing a replicated copy of the data and further comprise a number of sites from the plurality of sites across which to store a replicated copy of the data determined in accordance with a site replication factor.