Shuffle-Based Request Buffer for High-Volume Content Delivery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for live streaming media content and diverse device compatibility creates issues such as latency, resource overload, and potential malicious attacks, while managing multiple formats and configurations increases costs and complexity for media distribution services.
Innovation Solution
Implementing a shuffle-based request buffer to distribute requests uniformly across storage nodes, using shuffle sharding to isolate users and separate resource allocations, and splitting resource fleets for manifests and segments to optimize scaling and resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large number of users concurrently access the same resources, then content availability and access capacity increase, but latency increases and system reliability decreases due to resource overload and potential malicious attacks
Solution Approach 1:
The system segments the fleet of resources into multiple separate fleets, where each fleet is dedicated to handling a specific type of request (e.g., manifest requests, segment requests, authentication requests). This segmentation ensures that a spike in one type of request does not overload the entire system, thereby maintaining reliability while supporting high concurrency across different request types.
Solution Approach 2:
The patent introduces an intermediary layer (content delivery network with distributed cache servers) between users and the origin server. This intermediary absorbs and distributes request traffic, preventing direct overload on the origin server and protecting against malicious attacks while maintaining content availability.
2Adaptability or versatility
If separate records and files are maintained for various formats and configurations, then device compatibility and content format support improve, but cost and operational complexity increase
Solution Approach 1:
The patent implements a universal content delivery architecture where a single content delivery network serves multiple device types and formats through standardized interfaces. The distributed cache servers and content delivery logic are designed to handle various formats (HLS, DASH, progressive download) and device configurations through a unified system, reducing operational complexity while maintaining broad compatibility.
3Speed
If request traffic is concentrated on certain events or occurrences, then content delivery speed may improve, but latency increases and resource availability decreases due to traffic spikes
Solution Approach 1:
The system performs preliminary actions by pre-caching content at distributed cache servers before actual user requests arrive. During high-demand events, this pre-positioned content can be delivered immediately without waiting for origin server processing, thereby maintaining fast delivery speeds while avoiding latency caused by traffic spikes to the origin.
Solution Approach 2:
The patent distributes content delivery across multiple dimensions - geographically distributed cache servers, multiple content delivery fleets, and various delivery protocols (HLS, DASH, progressive). This multi-dimensional distribution prevents any single point from becoming a bottleneck during traffic spikes, maintaining both speed and low latency.
Data Source
AI summary
Approaches are disclosed for managing aspects of content delivery in a multi-tenant environment. A request buffer can be used to remove correlations between requests and randomly shuffle requests without storing all the requests concurrently. A shuffle sharding algorithm can be used to randomly allocate a subset of resources to different users in order to ensure less than a maximum risk of one user impacting the use of all resources allocated to other users. In some embodiments, separate fleets of resources can be maintained for manifests and video segments to allow for more accurate scaling and customization. Multiple manifests can also be associated with a single endpoint to allow multiple media players to obtain similar content segments from the single endpoint.