Hierarchical Cost-Based Caching for Online Media
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional caching algorithms for online media, particularly in SSD caches, fail to effectively manage the rapid changes in popularity of assets and do not consider the cost associated with re-obtaining evicted assets, leading to inefficiencies in hit-rate, replacement rate, and storage management.
Innovation Solution
A method for operating a cache device that calculates a marginal value based on a moving average and characteristic marginal value, considering the cost of assets and their popularity, to determine whether to store or evict assets, using a hierarchical caching structure with exponentially weighted moving averages and LRU eviction policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional caching algorithms (LRU, GDSF, LFU-DA) are used to manage cache storage, then implementation simplicity is maintained, but hit-rate and replacement rate performance deteriorate under rapid popularity changes
Solution Approach 1:
The patent transforms the caching decision from simple temporal recency (LRU) or frequency (LFU) metrics to a cost-aware parameter that incorporates both asset popularity and re-obtainment cost. The marginal value calculation MV(i) = (1/T_i) * log(C_i) dynamically adjusts caching priorities based on inter-arrival time T_i and cost C_i, allowing the system to adapt to rapid popularity changes while maintaining analytical tractability.
Solution Approach 2:
The patent introduces a feedback mechanism where the cache controller continuously monitors asset access patterns (inter-arrival times) and adjusts caching decisions based on calculated marginal values. The system learns from past access patterns and uses this information to make informed eviction and admission decisions, improving hit-rate over time without requiring complex machine learning models.
2Productivity
If cache size is increased to store more video chunks and quality levels, then hit-rate improves, but storage cost and system complexity increase
Solution Approach 1:
The patent applies local quality by treating different video assets and cache locations differently based on their specific characteristics. Instead of uniform caching policies, the system calculates individual marginal values for each asset based on its inter-arrival time and re-obtainment cost, allowing high-value assets to occupy premium cache space while low-value assets are evicted first. This localized decision-making maximizes cache utility without requiring proportional increases in cache size.
3Speed
If SSD memory is used for cache storage to improve access speed, then hit-rate performance improves, but the limited write cycle lifetime becomes a constraint
Solution Approach 1:
The patent converts the limitation of limited SSD write cycles into a beneficial constraint by incorporating re-obtainment cost into the caching decision framework. Assets with high re-obtainment costs (which would require expensive fast storage) are prioritized for caching, while assets with low re-obtainment costs can tolerate slower storage or re-fetching. This transforms the SSD's write cycle limitation into a mechanism for intelligent resource allocation.
4Productivity
If cache eviction policies consider re-obtainment cost, then replacement rate performance improves, but calculation complexity and processing overhead increase
Solution Approach 1:
The patent segments the caching decision into two manageable components: asset-specific factors (inter-arrival time T_i and re-obtainment cost C_i) and system-state factors (current cache utilization and marginal value thresholds). By separating these concerns and using pre-calculated logarithmic transformations, the system avoids complex real-time optimization while still achieving cost-aware eviction decisions. The segmentation allows independent calculation and comparison of marginal values without requiring full-system state analysis.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method of operating a first cache device may include receiving, at the first cache device, a request to send a first asset to a second device; determining whether the first asset is stored at the first cache device; and when the determining determines that the first asset is not stored at the first cache device, obtaining, at the first cache device, the first asset, comparing, at the first cache device, a moving average of a marginal value of the first asset with respect to the first cache device and a characteristic marginal value of the first cache device, calculating a cost associated with the first asset, selectively storing the first asset at the first cache device based on the comparison, and sending the obtained first asset and the calculated cost to the second device.