Edge Router Cache Eviction for Streaming Stall Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Over-the-top (OTT) streaming systems face long tail latency issues in content distribution networks (CDNs), leading to increased stall duration tail probability (SDTP), which negatively impacts user experience by delaying multimedia content playback.
Innovation Solution
Implementing a probabilistic scheduling approach in edge routers that models each cache server and content stream as separate queues, allowing for the characterization of download and playback times, and evicting multimedia chunk files from cache based on a threshold period since the last request, prioritizing files with soonest expiration to maintain cache capacity and minimize SDTP.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional cache management is used in CDN edge routers, then cache capacity is maintained, but stall duration tail probability increases due to long tail latency
Solution Approach 1:
The system performs preliminary actions by pre-fetching and caching multimedia chunk files before they are requested by users. The probabilistic scheduler predicts which files will be requested soon and proactively loads them into cache, reducing the likelihood of stalls when actual requests occur.
Solution Approach 2:
The cache management system dynamically adjusts its behavior based on real-time conditions. The probabilistic scheduler continuously monitors cache usage patterns, request rates, and file expiration times, adapting the eviction and pre-fetching decisions to current system state and demand patterns.
2Reliability
If cache capacity is increased to reduce stalls, then content delivery reliability improves, but system complexity and cost increase
Solution Approach 1:
The system changes key parameters of cache management by transitioning from deterministic eviction policies to probabilistic scheduling. Instead of using fixed rules like LRU (Least Recently Used), the system employs probability-based decisions that consider multiple factors including request patterns, file popularity, and cache age, optimizing performance without requiring larger cache capacities.
Solution Approach 2:
The probabilistic scheduler implements feedback mechanisms by continuously monitoring cache hit rates, request patterns, and stall durations. This feedback is used to adjust scheduling decisions and improve predictions over time, enabling effective cache management with existing capacity rather than requiring expansion.
3Reliability
If probabilistic scheduling is implemented to minimize SDTP, then user experience improves, but processing complexity at edge routers increases
Solution Approach 1:
The system segments the cache management function into distinct components: a probabilistic scheduler that makes high-level decisions about which files to pre-fetch and retain, and simpler eviction policies that handle routine cache operations. This segmentation allows the complex probabilistic logic to operate at appropriate granularity while keeping edge router processing manageable.
Data Source
AI summary
A method executed by a processing system of an edge router deployed in a content distribution network includes receiving a request from an endpoint device for a first file, determining that a portion of the first file is not stored in a cache of the edge router, determining that the cache is at a capacity threshold, selecting a second file to evict from the cache, wherein the second file is one of a plurality of files stored in the cache, wherein each file of the plurality of files is scheduled to be evicted from the cache when a threshold period of time has passed since a last request for the each file was received by the edge router, and wherein the threshold period of time associated with the second file is scheduled to expire soonest among all of the plurality of files, and evicting the second file from the cache.