Scalable Object Storage via Ring-Based Partitioning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional file storage systems face challenges in scalability and data safety when expanding storage capabilities in cloud computing environments, leading to resource inefficiencies and potential data loss due to centralization issues.

Innovation Solution

A massively scalable object storage system is implemented, utilizing a network of physical storage pools with availability zones, abstract partitions, and multi-master replication to ensure data redundancy and fault tolerance, along with a ring structure for efficient data management and synchronization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If data is centralized in a single storage system, then data management is simplified, but scalability is limited and data loss risk increases

Engineering Contradiction:

ImprovescalabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The storage system is divided into multiple storage pools distributed across different availability zones, with data further segmented into partitions that are independently managed. This segmentation enables scalable expansion by adding new pools without requiring system-wide reconfiguration, while maintaining simplified management through modular architecture.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system introduces a multi-dimensional organization structure with storage pools arranged in availability zones, partitions within pools, and replicas across zones. This dimensional hierarchy transforms the single-point storage model into a distributed lattice structure, enabling scalability while maintaining manageable complexity through layered abstraction.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Quantity of substance

If storage capacity is expanded by adding more nodes, then storage capability increases, but resource overhead and complexity increase

Engineering Contradiction:

Improvestorage capacityVSAvoidresource overhead

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

Each storage pool is designed as a universal unit that can serve multiple functions: storing different partitions, providing replicas for various data sets, and operating independently across availability zones. This multi-functionality allows the system to expand capacity by adding identical universal pools rather than specialized components, reducing resource overhead.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system dynamically adjusts parameters such as partition assignments, replica counts, and pool configurations based on storage capacity needs rather than structural complexity. When expanding capacity, only numerical parameters change (adding more pools and partitions) while the fundamental system architecture remains constant, minimizing the increase in operational complexity.

Inventive Principle:

Principle #35Parameter changes

3Reliability

If replicas are distributed across multiple availability zones, then data safety and fault tolerance improve, but system complexity increases

Engineering Contradiction:

Improvedata safetyVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

Each availability zone is designed with local quality characteristics, where storage pools within a zone share common failure modes but are isolated from failures in other zones. This local quality approach ensures data safety by distributing replicas across zones with different failure characteristics, while maintaining manageable complexity through zone-level abstraction that hides inter-zone coordination details.

Inventive Principle:

Principle #3Local quality

4Reliability

If data is distributed across multiple storage pools, then fault tolerance improves, but data management complexity increases

Engineering Contradiction:

Improvefault toleranceVSAvoiddata management

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The partition abstraction serves as an intermediary layer between data management operations and the distributed storage pools. Instead of directly managing individual pools or replicas, operations target partitions which automatically route to appropriate pools across availability zones. This intermediary simplifies data management by providing a unified access interface while maintaining fault tolerance through the underlying distributed architecture.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS9898521B2Massively scalable object storage system

Publication Date: 2018.02.20 RACKSPACE US

AI summary

Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system.