Deduplicated Cache Indexing with Exclusion Lists
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage architectures face challenges in balancing cost, performance, and capacity, particularly in building a tiered infrastructure for Protection Integrated Primary (PIP) storage systems, where SSDs offer high performance but high cost and limited write cycles, and disk storage provides lower performance but higher capacity, necessitating an efficient deduplication and compression strategy.
Innovation Solution
A deduplicated and compressed storage device system that utilizes a fingerprint index and file index to identify and store cached data efficiently, employing a write-evict unit (WEU) to extend SSD lifespan, and implementing a cache management layer with efficient insertion and eviction mechanisms to maintain a small fingerprint index and optimize cache performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If SSD is used for cache storage, then performance (IOPS) is improved, but cost and limited write cycles are worsened
Solution Approach 1:
The patent segments the cache into multiple regions (hot cache region and cold cache region) with different retention policies. The hot cache region stores frequently accessed data with shorter retention, while the cold cache region stores less frequently accessed data with longer retention. This segmentation allows the system to optimize write cycle usage by evicting data from the hot cache region more frequently, preventing excessive writes to the same SSD regions.
Solution Approach 2:
The patent implements periodic cache eviction and refresh operations. The cache manager periodically evaluates cache hits and evicts data from the hot cache region based on access patterns. This periodic action distributes write operations over time rather than concentrating them, thereby extending SSD lifespan while maintaining performance.
2Quantity of substance
If deduplication and compression are implemented, then cache capacity is increased, but processing complexity is worsened
Solution Approach 1:
The patent applies deduplication and compression operations in advance during data ingestion into the cache. Data is deduplicated against existing cache contents and compressed before being written to the SSD. This preliminary action reduces the actual data volume stored, increasing effective cache capacity while avoiding the need for complex real-time processing during cache operations.
Solution Approach 2:
The patent uses fingerprint-based copying to identify duplicate data blocks. Instead of storing redundant copies of identical data, the system creates fingerprints (hashes) of data blocks and stores only unique blocks. This copying approach efficiently increases cache capacity by eliminating duplicates while keeping the processing mechanism relatively simple through hash-based identification.
3Productivity
If a small fingerprint index is maintained, then cache performance is improved, but deduplication accuracy is worsened
Solution Approach 1:
The patent implements a two-region cache structure where the hot cache region maintains a comprehensive fingerprint index for accurate deduplication, while the cold cache region uses a simplified or partial indexing approach. This local quality differentiation allows the system to optimize cache performance by having fast access paths for hot data while maintaining adequate deduplication accuracy where it matters most (in the frequently accessed hot region).
Data Source
AI summary
A computer-implemented method for indexing content stored in a cache memory device is disclosed. The method starts with maintaining a file index having a plurality of extent entries, each extent entry corresponding to one of a plurality of file extents stored in a cache memory device that caches data stored in a persistent storage device of a storage system. In response to receiving a request to read a first file region of a first file, the method continues with retrieving first data block from the persistent storage device that contains the first data block, caching the first data block at a first storage location of the cache memory device; and creating a first extent entry in the file index having at least a first node, where the first node includes an address of the first storage location and a first bitmap indicating which data blocks are valid.