Multi-region data mirroring for cloud object storage durability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud object storage platforms face a significant risk of object loss as the number of objects increases, despite high durability guarantees, due to correlated failures within regions, which undermines the effectiveness of single region erasure coding.
Innovation Solution
Implementing data mirroring across multiple regions, with a master-slave garbage collection process, to maintain duplicate copies of data objects and minimize network bandwidth costs during garbage collection, allowing for increased durability without requiring independent object failures within regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is stored in a single region with erasure coding, then storage cost is reduced, but object durability decreases due to correlated failures
Solution Approach 1:
The patent segments data objects into multiple parts and distributes them across different regions. Instead of storing complete objects in a single region, the system divides objects into segments and places them in multiple geographic regions, thereby reducing the impact of region-specific failures while maintaining cost efficiency.
Solution Approach 2:
The patent introduces a geographic dimension to data storage by distributing objects across multiple regions. This transforms the storage architecture from a single-location system to a multi-dimensional distributed system, where data availability is improved by leveraging spatial distribution without proportionally increasing storage costs.
2Reliability
If data is mirrored across multiple regions, then object durability increases, but network bandwidth costs increase during garbage collection
Solution Approach 1:
The patent implements region-specific garbage collection where each region independently manages its own data segments. This local quality approach allows garbage collection to be performed locally without requiring constant cross-region communication, thereby reducing network bandwidth costs while maintaining the durability benefits of multi-region storage.
Solution Approach 2:
Each region performs self-service garbage collection on its own data segments without requiring centralized coordination or data transfer from other regions. This autonomous approach eliminates the need for expensive cross-region network operations during garbage collection while preserving data durability through distributed redundancy.
3Quantity of substance
If the number of objects stored increases, then storage capacity increases, but the likelihood of at least one object loss increases
Solution Approach 1:
The patent segments the large number of objects into smaller groups distributed across multiple regions. By dividing the storage portfolio into segmented units across different geographic locations, the system limits the propagation of failures and reduces the overall likelihood of object loss even as total storage capacity scales.
Solution Approach 2:
The patent adds geographic distribution as an additional dimension to storage capacity management. This allows the system to scale storage capacity across multiple regions rather than concentrating all objects in a single location, thereby increasing capacity while simultaneously reducing risk through spatial diversification.
Data Source
AI summary
Techniques for using data mirroring across regions to reduce the likelihood of losing objects in a cloud object storage platform are provided. In one set of embodiments, a computer system can upload first and second copies of a data object to first and second regions of the cloud object storage platform respectively, where the first and second copies are identical. The computer system can then attempt to read the first copy of the data object from the first region. If the read attempt fails, the computer system can retrieve the second copy of the data object from the second region.