Metadata Re-balancing in Storage Expansion Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scale-out storage systems face challenges in re-balancing metadata across newly added storage nodes during expansion, requiring minimal interruption to ongoing storage processes and maintaining performance.
Innovation Solution
A method and system for metadata re-balancing that involves an expansion manager coordinating compute entities to update lock and mapping metadata structures across new storage nodes using translation functions, allowing seamless transition to expanded metadata storage space without interrupting storage operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If metadata re-balancing is performed across newly added storage nodes during expansion, then storage capacity and performance are improved, but system interruption and latency increase
Solution Approach 1:
The patent applies preliminary action by pre-calculating translation functions and preparing metadata mapping structures before the expansion operation. The system computes new metadata addresses and updates translation functions in advance, so that when storage nodes are added, the metadata re-balancing can proceed smoothly without interrupting storage operations. This allows the system to expand capacity while minimizing system interruption.
2Adaptability or versatility
If metadata re-balancing is performed during storage expansion, then storage resources are optimized, but deadlock risk and system complexity increase
Solution Approach 1:
The patent applies segmentation by dividing the metadata re-balancing process into distinct phases: lock metadata updating, mapping metadata updating, and translation function updating. Each phase is handled separately with specific compute entities responsible for specific tasks. This segmentation reduces system complexity by breaking down the complex re-balancing operation into manageable, coordinated steps while still achieving optimal storage resource utilization.
3Speed
If translation functions are updated for new storage nodes, then metadata access efficiency is improved, but computation overhead and latency increase
Solution Approach 1:
The patent applies preliminary action by pre-computing translation functions and storing them in metadata structures before expansion operations. The system calculates address mappings and updates translation functions in advance, so that during actual storage operations, metadata can be accessed efficiently without real-time computation overhead. This balances metadata access efficiency with acceptable computation timing.
Data Source
AI summary
A method that includes instructing multiple compute entities, by an expansion manager and during a first phase of the stored entity metadata re-balancing, to: lookup any stored entity metadata by using: (a) a current translation function for linking identifiers of stored entities to stored entities metadata; wherein the current translation function is based on a size of a current storage space allocated to stored entities metadata; and (b) a next translation function for linking identifiers of stored entities to stored entities metadata; wherein the next translation function is based on a size of a next storage space allocated to stored entities metadata; wherein the current space is expanded during the expansion of the storage system to provide the next storage space, and to update any stored entity metadata accessed using the current translation function without updating stored entity metadata accessed using the next translation function.