Two-Phase Cache Eviction with Hot Cold Ghost Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The clock cache replacement algorithm is not scan-friendly, resulting in high CPU overhead and memory usage, particularly during sequential data scans, where important memory pages are evicted while unimportant pages are retained.
Innovation Solution
A cache management data structure comprising a cold queue, a ghost queue, and a hot queue, where the hot queue is configured as a ring buffer with two bits per storage location, allowing for efficient data mapping and addition, reducing CPU overhead and memory usage by avoiding unnecessary evictions during sequential scans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the clock cache replacement algorithm is used, then cache replacement is performed, but important memory pages are evicted while unimportant pages are retained during sequential scans
Solution Approach 1:
The cache is segmented into multiple regions: a hot queue for frequently accessed pages, a cold queue for infrequently accessed pages, and a ghost queue for recently evicted pages. This segmentation allows the system to retain important pages in the hot queue while efficiently evicting unimportant pages to the cold queue during sequential scans, resolving the contradiction between replacement accuracy and scan efficiency.
Solution Approach 2:
The system performs preliminary classification of memory pages into hot, cold, and ghost queues based on access patterns before sequential scans occur. By pre-organizing pages in the hot queue with high-priority items and the cold queue with low-priority items, the system ensures that during sequential scans, only unimportant pages are evicted while important pages remain cached, improving both replacement accuracy and scan efficiency.
2Ease of operation
If the clock cache replacement algorithm is used, then cache management is performed, but CPU overhead is high
Solution Approach 1:
The cache management system operates autonomously by automatically classifying pages into hot, cold, and ghost queues based on access patterns without requiring intensive CPU intervention. The ring buffer structure and bit manipulation operations enable the system to self-manage cache replacement with minimal CPU overhead, maintaining ease of operation while reducing energy consumption.
Solution Approach 2:
The system uses simple bit flags (two bits per storage location in the hot queue) to track page status and priority, changing these parameters efficiently to manage cache replacement. This parameter-based approach allows for lightweight cache management operations that minimize CPU overhead while maintaining full cache management capability.
3Quantity of substance
If the clock cache replacement algorithm is used, then cache replacement is performed, but memory usage is high
Solution Approach 1:
The cache memory is divided into distinct segments: hot queue, cold queue, and ghost queue, each serving a specific purpose. This segmentation allows efficient utilization of memory capacity by storing different types of pages in appropriate regions, maximizing the useful memory capacity while keeping the overall structure manageable through clear functional separation.
Solution Approach 2:
The hot queue is implemented as a ring buffer with compact storage locations, efficiently utilizing memory space. The ghost queue stores only location identifiers rather than full page data, reducing memory usage while maintaining the ability to track recently evicted pages. This copying and reference approach optimizes memory capacity utilization.
Data Source
AI summary
The present disclosure provides techniques for managing a cache of a computer system using a cache management data structure. The cache management data structure includes a cold queue, a ghost queue, and a hot queue. The techniques herein improve the functioning of the computer because management of the cache management data structure can be performed in parallel with multiple cores or multiple processors, because a sequential scan will only pollute (i.e., add unimportant memory pages) cold queue, and to an extent, ghost queue, but not hot queue, and also because the cache management data structure has lower memory requirements and lower CPU overhead on cache hit than some prior art algorithms.