Multi-Layered Cache Metadata Structure for Clustered Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing clustered storage systems face challenges in efficiently accessing and managing cache memory across multiple storage devices, leading to slower data access times and increased memory requirements for cache metadata.
Innovation Solution
A method and apparatus for a clustered storage system that utilizes a multi-layered cache metadata structure to quickly locate cached data, reducing the number of remote direct memory access (RDMA) operations and memory space needed for cache metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a traditional cache metadata structure is used in clustered storage systems, then the system can maintain cache functionality, but data access time increases and memory space requirements for cache metadata increase
Solution Approach 1:
The cache metadata structure is segmented into multiple layers (first layer metadata and second layer metadata), where each layer serves a specific function in the address translation process. This segmentation allows the system to reduce the size of individual metadata structures while maintaining overall functionality, thereby reducing total memory space requirements.
Solution Approach 2:
The patent introduces a hierarchical dimension to the cache metadata structure by organizing metadata into multiple layers. This dimensional change transforms a flat, single-layer metadata structure into a multi-layer hierarchical structure, enabling more efficient memory utilization and faster access paths.
2Productivity
If a traditional cache metadata structure is used in clustered storage systems, then the system can maintain cache functionality, but the number of RDMA operations increases
Solution Approach 1:
The first layer metadata is prepared and cached in advance, containing translation information that can be quickly accessed without requiring RDMA operations. This preliminary preparation of metadata reduces the need for repeated RDMA operations during data access, thereby improving system efficiency.
3Adaptability or versatility
If cache memory is distributed across multiple storage devices, then the system can scale memory capacity, but data access complexity increases
Solution Approach 1:
The multi-layer cache metadata structure acts as an intermediary mechanism that simplifies access to distributed cache memory. The hierarchical metadata provides a standardized interface for translating logical addresses to physical locations across distributed storage devices, reducing access complexity despite the distributed architecture.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
A clustered storage system includes a plurality of storage devices, each of which contributes a portion of its memory to form a global cache of the clustered storage system that is accessible by the plurality of storage devices. Cache metadata for accessing the global cache may be organized in a multi-layered structure. In one embodiment, multi-layered structure has a first layer first including a first address array, and the first address array include addresses pointing to a plurality of second address arrays in a second layer. Each second address array in the second layer includes addresses, each of which points to data that has been cached in the global cache.