Cache Replacement Policy Using Timestamp Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems face inefficiencies in deallocation of cached data pages due to the need for frequent exclusive locks in least-recently used (LRU) policies, leading to contention and performance issues when adjusting cache size.
Innovation Solution
Implementing a resource queue with timestamp-based ordering and resource dispositions to manage cached objects, allowing for efficient deallocation of resources without frequent lock modifications, and supporting variable-sized caches with fine-grained control over cache lifetimes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a least-recently used (LRU) policy is used to manage cache, then data pages can be deallocated based on usage time, but lock contention increases significantly due to frequent list modifications
Solution Approach 1:
The patent segments the cache management into two independent parts: a skip list that tracks resource usage order (read-only during deallocation) and a resource queue that manages allocation/deallocation. This segmentation allows concurrent access to the skip list while modifications occur only in the resource queue, eliminating lock contention between tracking and deallocation operations.
Solution Approach 2:
The patent performs preliminary actions by pre-building the skip list structure during resource allocation and usage, so that when deallocation is needed, the ordering information is already available. Resources are inserted into the skip list in chronological order of their last usage, eliminating the need for frequent list modifications during deallocation operations.
2Ease of operation
If a conventional LRU policy with linked list is used, then deallocation can be performed, but modification of the list requires exclusive locks causing contention
Solution Approach 1:
The patent introduces a resource queue as an intermediary data structure that mediates between the skip list (which maintains usage order) and the actual cache resources. The resource queue acts as a buffer that can be modified without affecting the skip list, allowing deallocation operations to proceed without acquiring locks on the usage-tracking structure.
Solution Approach 2:
The patent replaces the mechanical linked list modification system with a timestamp-based skip list system. Instead of physically moving nodes in a linked list during deallocation, the system uses pre-computed timestamps and skip list traversal to identify resources for deallocation, substituting mechanical list operations with more efficient data structure operations.
3Speed
If cache size is increased to improve performance, then faster access is achieved, but memory usage increases and may exceed available resources
Solution Approach 1:
The patent implements dynamic cache management where the cache size and composition can change over time based on usage patterns. The system continuously monitors resource usage timestamps and automatically deallocates least-recently-used resources when memory pressure occurs, allowing the cache to dynamically adapt its size and contents to balance performance and memory consumption.
Solution Approach 2:
The patent changes the parameter of cache resource selection from static (fixed-size cache or simple LRU) to dynamic based on multiple factors including usage timestamps, resource dispositions, and current memory availability. This allows the system to optimize cache utilization by keeping frequently accessed resources while evicting stale or low-priority resources.
4Reliability
If frequent lock acquisitions are used in LRU policy, then correct deallocation ordering is maintained, but performance degradation occurs due to contention
Solution Approach 1:
The patent creates a logical copy of the usage ordering information in the skip list structure, which is updated during resource allocation and usage but not modified during deallocation. This copying approach allows the system to maintain correct deallocation ordering by traversing the immutable skip list structure without acquiring locks, separating the read operation (for ordering) from the write operation (deallocation).
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system includes creation of a first resource queue indicating an order of resources stored in a memory, the order based on respective timestamps associated with the stored resources, association of the first resource queue with a first queue timestamp, reception of a first command to deallocate a first amount of stored resources from the memory, determination that a first stored resource indicated by the first resource queue is associated with a timestamp earlier than the first queue timestamp, deallocation of the first stored resource from the memory, reception of a second command to deallocate a second amount of stored resources from the memory, determination that the first resource queue indicates no stored resources which are associated with a timestamp earlier than the first queue timestamp, and, in response to the determination that the first resource queue indicates no stored resources which are associated with a timestamp earlier than the first queue timestamp, creation of a second resource queue indicating a second order of second resources stored in the memory, the second order based on respective second timestamps associated with the stored second resources.