Multi-grained Writeback Cache with Fine Block Descriptors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face challenges in maintaining consistency and scalability when handling high workload density demands, particularly with small random writes, due to difficulties in scaling metadata memory footprint and ensuring consistency across replicated writes in distributed storage systems.
Innovation Solution
A multi-grained metadata model with a cache management mechanism that utilizes fine block descriptors (FBDs) for cache storage and coarse block descriptors (CBDs) for back-end storage, allowing for variable-size allocation and efficient destaging, thereby maximizing NVM utilization and reducing memory footprint.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If conventional caching systems are used to handle small random writes, then I/O latency is improved, but it is difficult to maintain consistency across replicated writes and scale metadata memory footprint
Solution Approach 1:
The patent segments the metadata structure into hierarchical levels (global namespace, volume namespace, file namespace) that can be independently managed and scaled. This segmentation allows the system to handle small random writes with low latency while maintaining consistency across replicated writes by distributing metadata management across multiple manageable segments rather than a monolithic structure.
2Quantity of substance
If the cache address space is scaled to tens to hundreds of terabytes, then fine-grained updates are supported, but the metadata memory footprint becomes difficult to scale
Solution Approach 1:
The patent implements a nested hierarchical metadata structure where global namespace entries contain references to volume namespaces, which in turn contain file namespace entries. This nesting allows the system to manage tens to hundreds of terabytes of cache address space by organizing metadata in nested containers, reducing the memory footprint compared to a flat metadata structure that would require entries for every possible cache location.
Solution Approach 2:
The patent transitions from a one-dimensional flat metadata array to a multi-dimensional hierarchical namespace structure. This dimensional change allows the system to address large cache spaces efficiently by organizing metadata across multiple hierarchical levels (global→volume→file), reducing the memory required to track and manage large quantities of data while maintaining fine-grained update capabilities.
3Quantity of substance
If back-end address space is configured for petabytes, then coarse-grained updates are supported, but scaling becomes difficult with conventional systems
Solution Approach 1:
The patent creates a universal hierarchical namespace structure that serves multiple functions simultaneously: it supports both fine-grained updates for cache operations and coarse-grained updates for back-end operations. The same namespace hierarchy can be used to manage both small random writes to cache and large sequential writes to back-end storage, eliminating the need for separate metadata management systems and improving scalability across petabyte-scale address spaces.
Data Source
AI summary
In one embodiment, a system includes a cache storage device and a processor and logic integrated with and/or executable by the processor. The logic is configured to receive a write request including data to be written to the cache storage device. The logic is also configured to determine a size of the write request. Moreover, the logic is configured to select a chunk size from among a plurality of chunk sizes designated for storing data in the cache storage device. In addition, the logic is configured to allocate a fine block descriptor (FBD) having the selected chunk size to the write request.