Chunk Placement Indexing for Faster Node FTT Rebalancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems face inefficiencies in rebalancing data chunks due to the need for repeated scanning and calculation of node Failure To Tolerate (FTT) across all chunks, which affects performance and the ability to quickly determine minimum node FTT during maintenance operations.
Innovation Solution
A method and system that tracks node FTT by adding FTT task entries to a chunk index, including chunk identifiers and optimized FTT, allowing for efficient rebalancing of block placements without scanning all chunks, and quickly determining minimum node FTT for maintenance safety.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional storage systems scan all chunks to calculate node FTT for rebalancing, then accurate node FTT determination is achieved, but system performance deteriorates and maintenance operations are delayed
Solution Approach 1:
The system pre-calculates and stores node FTT values in FTT task entries within the chunk index during data placement operations. This preliminary action eliminates the need for repeated scanning and calculation during maintenance operations, as the node FTT information is already available in the stored entries, thus resolving the contradiction between measurement precision and productivity.
2Measurement precision
If conventional storage systems scan all chunks during maintenance operations, then minimum node FTT is accurately determined, but maintenance time increases
Solution Approach 1:
The invention extracts node FTT calculation from the maintenance operation process itself and stores it separately in FTT task entries during normal data placement. During maintenance, only the pre-stored FTT task entries need to be queried to determine minimum node FTT, eliminating the time-consuming full scan of all chunks while maintaining accurate determination.
3Reliability
If repeated scanning and calculation of node FTT is performed, then current node FTT status is accurately tracked, but computational overhead increases
Solution Approach 1:
Instead of repeatedly scanning and calculating the actual node FTT status, the system creates and stores copies of FTT task entries in the chunk index that contain pre-calculated node FTT information. These entries serve as reliable records of the node FTT status without requiring repeated computational overhead to verify or update them during maintenance operations.
Data Source
AI summary
Data placement is adjusted. For instance, it is determined whether a node Failure To Tolerate (FTT) of a chunk in a storage system meets an optimized node FTT corresponding to the chunk, and, when it is determined that it does not meet, an FTT task entry for the chunk is added to a chunk index of the storage system. The added FTT task entry includes an identifier of the chunk, its node FTT, and the corresponding optimized system node FTT. Furthermore, instead of scanning placement information for each chunk, the storage system may adjust, based on the FTT task entry in the chunk index, placement of blocks of the corresponding chunk in system nodes. Therefore, the need to scan the placement information of all chunks and calculate their node FTTs in rebalance operations and minimum node FTT queries is eliminated, thereby improving the performance of the storage system.