Storage Cluster Authority Splitting for Metadata Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modular, expandable storage systems face delays and increased latency in accessing user data due to growing metadata, which can lead to performance deterioration, even with distributed processing in storage clusters.
Innovation Solution
The method involves distributing user data across a storage cluster by splitting a parent authority into child authorities and assigning wards to these authorities, allowing for efficient data management and access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If storage systems are expanded to accommodate more user data, then storage capacity increases, but metadata search time increases causing access latency
Solution Approach 1:
The patent divides the storage system into multiple storage clusters, each managing a portion of the total storage capacity. By segmenting the metadata management across multiple clusters rather than having a single centralized metadata structure, the system can search through smaller metadata portions in parallel, reducing access latency while maintaining expanded storage capacity.
2Productivity
If storage clusters are expanded with more processing power, then data access capability improves, but system complexity increases
Solution Approach 1:
The patent segments the storage system into multiple independent storage clusters with distributed processing. Each cluster operates semi-autonomously with its own processing capabilities, allowing the system to scale processing power by adding clusters rather than complicating a single centralized system. This modular approach improves data access capability while managing system complexity through standardized cluster units.
3Ease of operation
If metadata is centralized for easier management, then management simplicity improves, but search performance deteriorates
Solution Approach 1:
The patent segments metadata management across multiple storage clusters, where each cluster maintains its own metadata for the data it stores. This distributed metadata approach allows each cluster to manage its metadata independently (maintaining operational simplicity) while enabling parallel search operations across clusters (improving search speed), avoiding the performance deterioration of centralized metadata search.
Data Source
AI summary
A method for storage cluster expansion is provided. The method includes distributing user data throughout a storage cluster as directed by each of a plurality of authorities in the storage cluster. Each of the plurality of authorities has a plurality of wards, and each of the plurality of wards has ownership of a range of the user data. The method includes splitting one of the plurality of authorities, as a parent authority, into at least two child authorities and assigning a first subset of the plurality of wards of the parent authority to one of the at least two child authorities, and a second subset of the plurality of wards of the parent authority to another one of the at least two child authorities.