Scalable Object Storage via Ring Topology and Zone Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file storage systems face challenges in scalability and data safety when expanding storage capabilities in cloud computing environments, leading to resource inefficiencies and potential data loss due to centralization issues.
Innovation Solution
A scalable object storage system is implemented using multiple physical storage pools across different availability zones, with data replication and consistent hashing to ensure each replica is stored in a distinct zone, and a ring structure for efficient data management and rebalancing to maintain performance constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is stored in a centralized storage system, then data access is simplified, but scalability and fault tolerance deteriorate
Solution Approach 1:
The patent divides the storage system into multiple distributed storage nodes organized in a ring structure, where each node stores a portion of the data. This segmentation allows the system to scale by adding more nodes while maintaining data accessibility through the ring topology, where data can be retrieved from any node that has the required data replica.
2Reliability
If data is replicated across multiple storage pools, then data safety and availability are improved, but system complexity increases
Solution Approach 1:
The patent creates multiple replicas of data objects and distributes them across different storage pools in the ring structure. Each storage pool maintains copies of the same data, ensuring that if one pool fails, the data remains accessible from other pools. This copying approach simplifies the reliability mechanism compared to complex error correction codes while effectively protecting against data loss.
3Productivity
If storage capacity is expanded rapidly, then cloud service capabilities are enhanced, but resource efficiency and data safety are compromised
Solution Approach 1:
The patent implements a dynamic ring structure where storage nodes can be added or removed without disrupting the entire system. When new storage pools are added, the system dynamically rebalances data replicas across the expanded capacity. This dynamic approach allows rapid storage expansion while maintaining resource efficiency by utilizing all available capacity and avoiding the need to rebuild the entire storage system during expansion.
Data Source
AI summary
An example method for storing data includes providing a plurality of physical storage pools, each storage pool including a plurality of storage nodes coupled to a network. The method also includes mapping a partition of a plurality of partitions to a set of physical storage pools, where each physical storage pool of the set of physical storage pools is located in a different availability zone, and the storage nodes within an availability zone are subject to a correlated loss of access to stored data. The method further includes receiving a data management request over the network, the data management request being associated with a data object. The method also includes identifying a first partition of the plurality of partitions corresponding to the received data management request and manipulating the data object in the physical storage pools mapped to the first partition in accordance with the data management request.