Adaptive Cache Limits for Bursty Concurrent Transactions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cache eviction strategies struggle with sudden and frequent bursts of concurrent transactions, leading to memory overflow issues and performance penalties, particularly in cloud-based caching systems.
Innovation Solution
Implementing a dual cache size limit system with a primary and secondary limit, coupled with adaptive eviction policies and cache miss heuristics, to manage cache size dynamically and maintain performance during bursts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional eviction strategies are used to limit cache size, then memory overflow is prevented, but cache operations must be blocked leading to reduced throughput
Solution Approach 1:
The patent implements dynamic cache size limits with two thresholds (primary and extended) that adapt based on cache miss ratios. The system transitions between different eviction policies based on current cache size relative to these dynamic thresholds, allowing the cache to expand during low-traffic periods and contract during high-traffic periods without blocking operations
Solution Approach 2:
The system changes operational parameters by implementing different eviction policies based on cache size thresholds. When cache size is between primary and extended limits, a first eviction policy applies. When cache size exceeds the extended limit, a second stricter eviction policy activates. This parameter-based policy switching maintains reliability while preserving throughput
2Quantity of substance
If cache size is increased to handle burst transactions, then more data can be cached, but memory overflow occurs leading to performance penalties
Solution Approach 1:
The patent segments the cache size management into multiple zones defined by primary and extended cache size limits. This segmentation allows different eviction behaviors in different cache size zones, preventing uncontrolled growth while maintaining adequate capacity during normal operation
Solution Approach 2:
The extended cache limit acts as a cushion zone between the primary limit and memory overflow conditions. This beforehand cushioning allows the cache to temporarily exceed the primary limit during bursts without immediately triggering strict eviction, providing a buffer that prevents memory overflow while maintaining stability
3Reliability
If strict eviction policies are applied to control cache size, then cache size is kept in check, but all cache operations are blocked causing performance degradation
Solution Approach 1:
The patent applies different eviction policy strictness levels in different cache size contexts. When cache size is between primary and extended limits, a moderate eviction policy applies allowing operations to continue. When cache size exceeds the extended limit, a stricter policy activates. This local differentiation maintains size control while preserving operation continuity in normal conditions
Data Source
AI summary
A cache management system implements a primary cache limit and an extended cache size limit. A first cache policy is implemented for the cache size being between the primary cache limit and the extended cache limit. A second cache policy is implemented for the cache size being above the extended cache limit. The second cache policy is stricter than the first cache policy. In some implementations, the primary cache limit and the secondary cache limit are adapted up or down depending on Cache Miss ratio heuristics.