Data Storage Host Grouping Consolidation for Capacity Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage services face inefficiencies as storage hosts maintain metadata while emptying, leading to increased costs due to disproportionate data storage capacity usage as data becomes obsolete.
Innovation Solution
Implementing a data storage service that consolidates data from multiple hosts to a smaller subset using redundancy encoding schemes, transferring data fragments across hosts to maintain durability and update metadata to reflect empty hosts, allowing for efficient reuse of storage capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is deleted from storage hosts as it becomes obsolete, then storage capacity is freed up, but metadata is still maintained on empty hosts increasing storage costs
Solution Approach 1:
The patent consolidates data from multiple storage hosts into fewer hosts by evaluating data storage host groupings and transferring data fragments between hosts. This merging process eliminates empty hosts that were maintaining metadata, thereby reducing storage costs while maintaining full storage capacity utilization.
2Loss of energy
If data is consolidated from multiple hosts to fewer hosts, then storage cost is reduced, but data durability must be maintained
Solution Approach 1:
The system performs preliminary evaluation of data storage host groupings to identify consolidation opportunities before executing data transfers. This preliminary analysis ensures that data durability requirements are met by selecting appropriate target hosts and maintaining redundancy encoding schemes throughout the consolidation process.
Solution Approach 2:
The patent implements a feedback mechanism where the data storage service continuously evaluates the state of storage hosts and adjusts consolidation operations accordingly. This feedback loop ensures data durability is maintained by monitoring transfer progress and verifying data integrity throughout the consolidation process.
3Quantity of substance
If storage hosts are consolidated to a smaller subset, then storage capacity utilization is optimized, but system complexity increases
Solution Approach 1:
The patent segments the storage system into data storage host groupings that can be independently evaluated and consolidated. This segmentation allows the complex consolidation process to be broken down into manageable units, reducing overall system complexity while optimizing storage capacity utilization.
Data Source
AI summary
A data storage service distributes a plurality of data fragments corresponding to a data object among one or more data storage host groupings in a manner that avoids a possibility of correlated loss of multiple data fragments by consolidation of data of a data storage host grouping onto a single data storage host. The data storage service selects a data storage host grouping and determines an amount of used capacity for the selected data storage host grouping. If the selected grouping satisfies an emptiness threshold, the data storage service selects a data storage host from the grouping and consolidates one or more data sets of the grouping onto the selected data storage host. Subsequently, the data storage service updates metadata for each data storage host of the selected data storage host grouping to specify a location of data stored therein.