Scalable Object Storage via Ring-Based Data Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file storage systems face challenges in scalability and data safety, particularly in cloud computing environments, where rapid expansion is needed while ensuring seamless user experience and data integrity, often requiring substantial resources and facing issues with data centralization and scale.
Innovation Solution
A scalable object storage system is developed, utilizing a method that includes providing physical storage pools across multiple availability zones, mapping abstract partitions to these pools for fault-tolerant replication, and using a ring structure to manage and rebalance data storage efficiently, ensuring data availability and safety through distributed storage and replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is centralized in a single storage system, then data access and management is simplified, but scalability is limited and single points of failure increase
Solution Approach 1:
The storage system is segmented into multiple independent storage pools distributed across different availability zones. Each pool operates autonomously, allowing the system to scale by adding more pools without increasing overall system complexity. The segmentation into zones, pools, and partitions creates a hierarchical structure that manages complexity at each level.
Solution Approach 2:
The patent introduces multiple dimensions of organization: availability zones (geographic dimension), storage pools (logical dimension), and partitions (data dimension). This multi-dimensional approach allows scalability along each dimension independently, transforming a single-point-of-failure system into a distributed system where failure in one dimension does not affect others.
2Productivity
If storage capacity is expanded rapidly to meet cloud computing demands, then user needs are met, but data safety and integrity may be compromised
Solution Approach 1:
Data is replicated across multiple storage pools in different availability zones before any single pool becomes the sole source of truth. This preliminary replication ensures that even during rapid expansion or failure scenarios, data safety is maintained. The system performs safety checks and validation during the replication process.
Solution Approach 2:
The patent implements redundancy through cross-zone replication, creating a cushion against data loss before failures occur. Multiple copies of data are maintained across different availability zones, so if one zone experiences issues during expansion, other zones provide backup protection, cushioning the system against data safety compromises.
3Reliability
If replicas are distributed across multiple availability zones for fault tolerance, then data availability is improved, but system complexity and resource overhead increase
Solution Approach 1:
Each storage pool and partition automatically manages its own replication state through self-service mechanisms. The system autonomously monitors data availability, performs replication, and handles failover without requiring complex external management. This reduces replication management complexity by distributing intelligence to individual components.
Solution Approach 2:
The system implements feedback loops where storage pools continuously monitor data replication status, availability zone health, and data integrity. This feedback enables automatic adjustment of replication strategies and failover decisions, reducing the need for complex manual management while maintaining high data availability across zones.
4Reliability
If substantial resources are allocated to ensure data safety during expansion, then data integrity is maintained, but resource efficiency decreases
Solution Approach 1:
The system applies partial replication strategies where not all data requires full replication across all zones. Critical data receives excessive replication for maximum safety, while less critical data receives partial replication, optimizing resource efficiency. This selective approach maintains data integrity for essential data while reducing overall resource consumption.
Data Source
Figure 1a
Figure 1b
Figure 2
AI summary
Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system.