Distributed Storage Audit Replication for Zero-RPO Switchover
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems face challenges in efficiently synchronizing and managing data and audit configuration across multiple storage sites, leading to potential data loss and disruption during failures, especially for object storage protocols, as they lack synchronous replication and consistent configuration management.
Innovation Solution
Implementing synchronous replication of data and configuration across mirrored storage clusters, ensuring zero recovery time objective (RTO) and zero recovery point objective (RPO) by replicating object storage configuration data and audit logs, and managing switchover processes to maintain consistency and avoid data loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication is implemented across distributed storage clusters, then data consistency and business continuity are improved, but system complexity and implementation difficulty worsen
Solution Approach 1:
The system segments replication into two independent streams: data replication and configuration replication. Data replication handles object storage bucket contents, while configuration replication handles OSP configuration data including audit settings. This segmentation allows each stream to be optimized independently, reducing overall system complexity while maintaining data consistency across clusters.
Solution Approach 2:
The patent introduces configuration data as an intermediary element that mediates between primary and secondary clusters. By replicating OSP configuration data (including audit configuration) alongside data, the system ensures that the secondary cluster can assume the primary cluster's role seamlessly during failover, achieving reliability without requiring complex direct synchronization of all system parameters.
2Reliability
If synchronous replication of audit configuration data is implemented, then audit consistency across clusters is improved, but replication time and processing overhead worsen
Solution Approach 1:
The patent merges data replication and configuration replication into a unified synchronous replication framework. By combining these replication streams, the system ensures that both data and audit configuration are replicated together, maintaining their consistency without requiring separate synchronization processes. This merging eliminates the need for additional replication time that would result from separate processes.
Solution Approach 2:
The system maintains continuous replication of both data and configuration data in real-time, ensuring that the secondary cluster always has up-to-date copies. This continuous action eliminates batch processing delays and ensures that audit consistency is maintained without periodic synchronization overhead, as the replication is ongoing and incremental.
3Reliability
If synchronous replication is used to ensure zero RPO, then data loss prevention is improved, but system resource consumption and operational cost worsen
Solution Approach 1:
By segmenting replication into data and configuration components, the system optimizes resource usage for each stream. Configuration data (including audit settings) is typically much smaller than data objects, so replicating it synchronously consumes minimal additional resources. This segmentation allows zero RPO to be achieved for critical configuration data without proportionally increasing overall resource consumption.
Solution Approach 2:
The patent applies different replication qualities to different data types: full synchronous replication for configuration data (to ensure consistency) and standard data replication for object contents. This local quality approach ensures that critical audit configuration data receives the highest level of protection with appropriate resources, while not over-provisioning resources for all data types equally.
4Reliability
If audit jobs are initiated on both primary and secondary clusters during switchover, then audit continuity is improved, but configuration management complexity and data synchronization difficulty worsen
Solution Approach 1:
The system performs preliminary replication of audit configuration data before switchover occurs. By ensuring that the secondary cluster already has the audit configuration replicated and ready, the switchover process can simply activate the existing audit job on the secondary cluster rather than initiating and configuring a new one during the transition. This preliminary action simplifies configuration management during failover.
Solution Approach 2:
The patent uses configuration copying to replicate OSP audit configuration from the primary cluster to the secondary cluster. This copying mechanism ensures that the secondary cluster has an exact replica of the audit configuration, allowing audit jobs to continue seamlessly after switchover without requiring complex reconfiguration or manual intervention, thereby improving audit continuity while managing complexity through automated copying.
Data Source
AI summary
In one embodiment, a computer implemented method includes comprises storing objects in a first bucket and files in a second bucket of a first storage cluster of the distributed storage system, initiating an audit job on the first storage cluster, synchronously replicating audit configuration data and mirroring audit data (e.g., audit files, logs) from the first storage cluster to the second storage cluster, performing a switchover process from the first storage cluster to the second storage cluster, and initiating an audit job on the second storage cluster based on the audit configuration during the switchover process. The first storage cluster initially handles input/output operations for a software application before the switchover process.