Granular Sync Semi-Sync Storage Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data synchronization architectures in distributed storage systems face challenges in maintaining data consistency and availability across distant storage elements while adhering to strict recovery point objectives (RPO) and recovery time objectives (RTO), especially in scenarios requiring both full synchronization and semi-synchronization at a logical storage object granularity.
Innovation Solution
The implementation of a granular sync/semi-sync architecture that coordinates data changes between primary and secondary logical storage objects through storage cluster synchronization engines, intercepting filesystem requests, and propagating changes efficiently across clusters, allowing for both full synchronization and semi-synchronization configurations to ensure data consistency and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is synchronized across distant storage systems to avoid single catastrophe, then data availability and protection are improved, but synchronization time and data loss risk increase
Solution Approach 1:
The patent divides the storage system into multiple storage elements (primary and secondary) distributed across different sites, allowing data to be segmented and stored in geographically separated locations. This segmentation enables local data access while maintaining overall system availability, resolving the contradiction between distant data replication for reliability and synchronization time by allowing independent operation of segmented storage elements.
Solution Approach 2:
The patent implements preliminary actions by creating snapshots and preparing data in advance before failures occur. Storage elements maintain pre-synchronized data copies and use change set logs to track modifications, enabling rapid recovery without real-time synchronization delays, thus improving reliability while minimizing synchronization time impact.
2Manufacturing precision
If strict RPO and RTO requirements are enforced, then data consistency is improved, but system complexity and performance overhead increase
Solution Approach 1:
The patent introduces change set logs as intermediary structures that track data modifications between storage elements. These logs act as mediators that simplify the synchronization process by providing a single source of truth for changes, reducing the complexity of maintaining data consistency across distributed systems while ensuring strict RPO requirements are met.
Solution Approach 2:
The patent implements different synchronization modes (full sync and semi-sync) that allow parameter changes in synchronization behavior based on specific requirements. Full sync ensures complete data consistency with strict RPO, while semi-sync allows for performance optimization with relaxed consistency requirements, enabling the system to adapt to different operational contexts without increasing overall complexity.
3Manufacturing precision
If full synchronization is implemented across all storage elements, then data consistency is improved, but performance and response time deteriorate
Solution Approach 1:
The patent implements dynamic synchronization where storage elements can operate in different synchronization modes (full sync or semi-sync) based on their specific needs and operational context. This dynamic approach allows the system to optimize performance by using semi-sync for non-critical data while maintaining full sync for critical data, resolving the contradiction between consistency and performance through adaptive behavior.
Solution Approach 2:
The patent applies partial synchronization by implementing semi-sync mode where not all storage elements require complete data consistency. This partial action approach synchronizes only the necessary data portions or uses best-effort synchronization for non-critical elements, improving overall system performance while maintaining adequate data consistency for operational needs.
4Reliability
If synchronous replication is used to meet RPO requirements, then data protection is improved, but response time and availability during writes deteriorate
Solution Approach 1:
The patent uses preliminary action by maintaining change set logs that track data modifications before they need to be replicated. Storage elements prepare data for synchronization in advance and use these logs to efficiently apply changes, reducing the time impact of synchronous replication on write operations while maintaining strong data protection and RPO compliance.
Data Source
AI summary
Data consistency and availability can be provided at the granularity of logical storage objects in storage solutions that use storage virtualization in clustered storage environments. To ensure consistency of data across different storage elements, synchronization is performed across the different storage elements. Changes to data are synchronized across storage elements in different clusters by propagating the changes from a primary logical storage object to a secondary logical storage object. To satisfy the strictest RPOs while maintaining performance, change requests are intercepted prior to being sent to a filesystem that hosts the primary logical storage object and propagated to a different managing storage element associated with the secondary logical storage object.