Distributed Storage Data Distribution Maintenance Hierarchy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in maintaining data accessibility during maintenance periods, as data centers may become unavailable due to maintenance activities, leading to impaired user operations.
Innovation Solution
A method and system for distributing data across multiple maintenance units in a distributed storage system, using a maintenance hierarchy with active and inactive states, where chunks of files are distributed across multiple maintenance units to ensure accessibility even when some units are inactive, and the number of chunks on each storage device is restricted to maintain data availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a distributed storage system, then storage capacity and accessibility are improved, but data availability is impaired during maintenance periods when maintenance units are inactive
Solution Approach 1:
The storage system is divided into multiple maintenance units, each independently manageable. Data is segmented and distributed across these units, allowing maintenance to be performed on individual units without affecting the entire system's data availability.
Solution Approach 2:
The system dynamically adjusts data distribution parameters based on maintenance unit states. When a maintenance unit transitions to inactive state, the system modifies chunk allocation parameters to ensure data remains accessible from active units, maintaining data availability throughout maintenance operations.
2Reliability
If chunks are distributed across multiple maintenance units, then data accessibility during maintenance is improved, but system complexity increases
Solution Approach 1:
The chunk distribution strategy is dynamic rather than static. The system continuously monitors maintenance unit states and automatically adjusts data placement decisions. This dynamic approach simplifies management by using rule-based logic instead of complex manual coordination, as the system autonomously adapts to changing conditions.
Solution Approach 2:
The distributed storage system autonomously manages data redistribution during maintenance without requiring complex external coordination. The system self-adjusts by monitoring unit availability and automatically rerouting data access to active units, reducing the operational complexity of maintenance management.
3Reliability
If the number of chunks per storage device is restricted, then data availability during maintenance is maintained, but storage utilization efficiency decreases
Solution Approach 1:
The system dynamically changes chunk distribution parameters based on the number of active maintenance units. When fewer units are active, the system adjusts parameters to distribute chunks more efficiently across available capacity, optimizing storage utilization while maintaining the minimum required availability thresholds.
Solution Approach 2:
The system creates a protective data distribution environment that inherently maintains availability requirements. By embedding availability constraints into the distribution algorithm itself, the system automatically preserves data accessibility without requiring additional overhead or reducing effective storage capacity, thus maintaining efficiency while ensuring reliability.
Data Source
AI summary
A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.