Cache Reclamation via Transaction Log Extension on SSDs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems using only non-volatile random access memory (NVRAM) for transaction logs are limited in capacity and bandwidth, necessitating an expansion to utilize other non-volatile storage with fast data access characteristics, such as solid state drives (SSDs), to enhance write data storage and management.
Innovation Solution
The transaction log is extended to utilize both NVRAM and SSDs, with SSDs dynamically allocated for log extension when NVRAM is full, allowing for increased storage of write data and associated descriptors, and implementing a cache management system to efficiently manage dirty pages and free up cache space by evicting older pages to SSDs for persistent storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If only NVRAM is used for transaction logs, then data access speed is fast, but storage capacity is limited
Solution Approach 1:
The transaction log storage is segmented into two parts: NVRAM for frequently accessed log entries requiring fast access, and SSD for extended storage capacity. This segmentation allows the system to maintain fast access speeds for active transactions while expanding total storage capacity to handle larger workloads.
Solution Approach 2:
The system transitions from a single-dimension storage approach (only NVRAM) to a two-dimension storage hierarchy (NVRAM + SSD). This dimensional change enables the system to simultaneously achieve fast access speeds through NVRAM and expanded capacity through SSD, resolving the contradiction between speed and quantity.
2Productivity
If cache memory is expanded to store more dirty pages, then write operation throughput increases, but cache memory footprint increases
Solution Approach 1:
The system extracts dirty pages from the cache memory and relocates them to SSD storage when cache capacity is sufficient. This extraction allows the cache to maintain a manageable footprint while still supporting high write throughput by temporarily buffering data before final persistence to SSD.
Solution Approach 2:
The system performs preliminary caching of dirty pages in memory before final persistence to SSD. This preliminary action in the cache layer enables write operations to complete quickly without immediately writing to SSD, thereby increasing write throughput while controlling cache footprint through selective eviction to SSD.
3Ease of operation
If dirty pages are evicted from cache to SSD, then cache space is freed for reuse, but data persistence is required
Solution Approach 1:
Before evicting dirty pages from cache to SSD, the system performs a preliminary flush operation that writes the dirty pages to the transaction log on SSD. This preliminary action ensures data persistence is maintained while the cache page can then be safely marked as clean and reused, resolving the contradiction between cache space availability and data persistence.
Data Source
AI summary
In a log based system, a reclamation list of cache pages containing dirty data can be maintained and used. The cached dirty data pages identified by the reclamation list can be candidates for eviction from the cache. A cached dirty data page on the reclamation list can be persistently logged in a transaction log extension having storage allocated from a non-volatile solid state drive (SSD). The reclamation list can include descriptors of the cached dirty data pages prioritized for eviction in accordance with a least recently used (LRU) policy so that the LRU cached dirty data page can be selected for eviction. When a dirty data page which was evicted from the cache is reloaded into cache, the dirty data page can be added to the reclamation list by adding its page descriptor to the reclamation list. A dirty data page can be removed from the reclamation list once flushed.