Efficiency Sets for Distributed Storage Data Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data storage systems, determining the most efficient cluster for moving or importing volumes of data is challenging due to the need to calculate the precise amount of unique data blocks, which requires comparing block identifiers across multiple volumes, a process that is computationally intensive and memory-intensive.
Innovation Solution
The use of efficiency sets, which are generated by masking block identifiers to reduce computation time and memory requirements, allows for the estimation of unique data blocks by comparing masked sets, thereby approximating the dedupability and optimizing data placement across clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If block identifiers are compared precisely across multiple volumes to calculate unique data blocks, then measurement precision is improved, but use of energy and computing resources worsens
Solution Approach 1:
The patent divides the block identifier into multiple segments or fields, comparing only specific segments rather than the entire identifier. This segmentation allows for approximate comparison that reduces computational energy while maintaining sufficient precision for determining unique data blocks across volumes.
Solution Approach 2:
The patent performs partial comparison of block identifiers by focusing on key distinguishing segments rather than complete identifier comparison. This partial action approach provides adequate precision for uniqueness determination while significantly reducing the computational energy required for the comparison process.
2Measurement precision
If complete block identifiers are compared across volumes, then measurement precision is improved, but loss of time worsens
Solution Approach 1:
The block identifier is segmented into multiple parts, with only critical segments being compared to determine uniqueness. This segmentation enables faster comparison operations that reduce time loss while preserving sufficient measurement precision for accurate unique data block determination and timely migration decisions.
Solution Approach 2:
The patent implements partial comparison by examining only the most discriminative segments of block identifiers rather than complete identifiers. This partial action approach achieves acceptable measurement precision for uniqueness determination while dramatically reducing the time required for comparison operations.
3Measurement precision
If precise comparison of block identifiers is performed, then measurement precision is improved, but use of energy by stationary object worsens
Solution Approach 1:
The patent segments the block identifier comparison process, focusing computational energy on comparing only the most significant segments that determine uniqueness. This segmentation reduces the overall energy consumption of the stationary storage system while maintaining sufficient measurement precision for accurate dedupability estimation.
Solution Approach 2:
The system performs partial comparison of block identifiers by examining only key segments necessary for uniqueness determination. This partial action approach provides adequate measurement precision for dedupability estimation while significantly reducing the memory and processing energy required for the comparison operation.
Data Source
AI summary
Disclosed are systems, computer-readable mediums, and methods for efficiency sets in a distributed system. A first efficiency set is determined for a first volume of data. Determining the first efficiency set includes selecting block identifiers for data blocks of the first volume, where each block identifier is used to access a particular data block corresponding to the first volume. Determining the first efficiency set further includes applying a mask to the selected block identifiers to mask at least one bit of each selected block identifier. The first efficiency set is compared to a second efficiency set for a second data store, and based on the comparison, an amount of unique data blocks of the first volume is approximated.