Persistent Metadata Cache for Data Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face inefficiencies due to redundant operations and resource consumption in using caches and transaction logs for metadata management, leading to unnecessary caching and address translation, which wastes resources and complicates metadata processing.
Innovation Solution
Implementing a persistent metadata cache that allows direct caching and writing of metadata changes to internal volumes, eliminating the need for separate buffer caches and transaction logs, and enabling atomic metadata transactions, thus reducing redundant operations and improving resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate buffer caches and transaction logs are used for metadata management, then metadata consistency and durability are ensured, but system resources are wasted due to redundant operations and multiple caching instances
Solution Approach 1:
The patent combines the buffer cache and transaction log into a single persistent metadata cache structure. This unified cache eliminates redundant operations by storing metadata changes directly in a persistent format without requiring separate buffering and logging mechanisms, thereby reducing system resource consumption while maintaining metadata consistency through persistent storage semantics.
Solution Approach 2:
The persistent metadata cache serves multiple functions simultaneously: it acts as a buffer cache for metadata, a transaction log for durability, and a persistent storage layer. This multi-functional design eliminates the need for separate specialized components, reducing overall system resource usage while ensuring both consistency and durability through its persistent nature.
2Speed
If metadata is cached in both system cache and buffer cache, then metadata access speed is improved, but redundant operations increase and resources are wasted
Solution Approach 1:
The patent extracts the buffering function from the traditional two-cache architecture and integrates it directly into the persistent metadata cache. By eliminating the intermediate buffer cache layer, the system reduces caching operation complexity while maintaining fast access speeds through direct persistent storage access and avoidance of redundant copy operations.
3Manufacturing precision
If address translation is performed for each metadata flush operation, then correct mapping to storage volumes is achieved, but system resources are consumed repeatedly
Solution Approach 1:
The patent performs address translation preliminarily when metadata is initially written to the persistent metadata cache, rather than translating addresses repeatedly during each flush operation. This preliminary address mapping stores the correct storage volume addresses directly in the persistent cache structure, eliminating repeated translation operations and improving metadata processing efficiency while maintaining accurate address mapping.
Data Source
AI summary
A technique for storing metadata changes includes caching metadata changes in a persistent metadata cache. The persistent metadata cache is configured to cache metadata reads and writes directed to a set of internal volumes of the data storage system. A file system can access pages of the persistent metadata cache by specifying an identifier of an internal volume and an offset into that volume to which the metadata changes are directed.