Metadata Update Architecture Using Custom and Regular Buckets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data storage systems, frequently updated metadata pages consume excessive system resources and time due to the large number of updates, leading to performance degradation and resource inefficiency.
Innovation Solution
Implementing a metadata update architecture that distinguishes between regular and custom buckets, where custom buckets are optimized for high-frequency updates, using hash-based and non-hash-based organizations respectively, and applying updates in-place for counters, reducing the need for multiple entries and improving access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a unified metadata update structure is used for all metadata pages, then the system structure is simple and easy to maintain, but frequently updated metadata pages consume excessive system resources and time
Solution Approach 1:
The patent segments the metadata update structure into two distinct types: regular buckets for infrequently updated metadata pages and custom buckets for frequently updated metadata pages. This segmentation allows the system to apply different optimization strategies to different metadata pages, reducing resource consumption for high-frequency updates while maintaining simplicity for low-frequency updates.
Solution Approach 2:
The patent applies local quality by making custom buckets location-aware and optimized for specific high-frequency metadata pages, while regular buckets use a general hash-based organization. This allows the system to optimize resource usage locally at the frequently updated pages without complicating the entire metadata update structure.
2Speed
If hash-based organization is used for all metadata pages, then access efficiency is improved, but the overhead for frequently updated pages increases due to multiple entries
Solution Approach 1:
The patent changes the organizational parameters for custom buckets by making them location-aware and using non-hash-based organization methods. This parameter change reduces the overhead for frequently updated pages by eliminating the need for multiple hash-based entries while maintaining fast access through direct location-based routing.
3Reliability
If in-place updates are not applied, then data integrity is maintained through separate entries, but the number of entries increases and access efficiency decreases
Solution Approach 1:
The patent introduces dynamic update strategies where the update method adapts based on the metadata page type. For custom buckets with frequently updated pages, in-place updates are applied to improve efficiency. The system dynamically manages the update approach based on the specific characteristics of each metadata page, balancing data integrity with processing efficiency.
Data Source
AI summary
In at least one embodiment, processing can include receiving a metadata (MD) update for a MD page, wherein the MD update has an associated type; determining whether the type is a custom MD type; responsive to determining the type is a custom MD type, performing first processing including: determining a custom bucket handle for the MD update based at least in part on the type and a logical index (LI) uniquely identifying the MD page; and applying the MD update to a custom bucket associated with the custom bucket handle; and responsive to determining the type is a custom MD type, performing second processing including: determining a regular bucket handle for the MD update based at least in part on the LI of the MD page; and applying the MD update to a regular bucket associated with the regular bucket handle.