Distributed Live Stream Storage Management via Edge Server Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Central storage clusters in content delivery networks (CDNs) face bottlenecks due to overhead in managing streaming content, particularly in simultaneous uploads and deletions, which can degrade streaming performance as the number of streams increases, and existing solutions like central databases offer limited scalability.
Innovation Solution
Distributed management of live stream storage offloads segment expiration and deletion overhead from the central storage cluster to edge servers, using a file system-based implementation where edge servers track and manage segment expiration times, eliminating the need for a central database and reducing network connection overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a central storage cluster is used to aggregate streams uploaded through various CDN PoPs, then stream distribution capability is improved, but streaming performance degrades due to bottlenecks in managing simultaneous uploads and deletions
Solution Approach 1:
The patent segments the centralized management functions by distributing expiration tracking and deletion operations to individual edge servers. Each edge server independently manages its own uploaded segments' expiration times, eliminating the bottleneck of centralized management while preserving the centralized storage architecture for actual stream data.
Solution Approach 2:
The patent extracts the management overhead (tracking and deletion operations) from the central storage cluster and places it at the edge servers. This separation allows the central storage to focus on efficient data operations while edge servers handle the administrative burden of expiration management locally.
2Productivity
If additional hardware and resources are provisioned to offset increased loads on the central storage cluster, then streaming performance is maintained, but system complexity and cost increase
Solution Approach 1:
The patent implements self-service by enabling edge servers to autonomously track expiration times and perform deletions of their own uploaded segments. Each edge server maintains local records of segment expiration times and independently executes deletion operations, eliminating the need for additional centralized management infrastructure.
3Ease of operation
If a central database is used to track stream segments and manage deletions, then management overhead is reduced, but scalability is limited and bottlenecks occur at high stream volumes
Solution Approach 1:
The patent segments the centralized database management approach by distributing the tracking function to individual edge servers. Instead of one central database handling all segments, each edge server maintains its own local tracking for segments it uploaded, enabling linear scalability as more edge servers are added to the CDN network.
4Productivity
If edge servers independently track and manage segment expiration, then central storage cluster bottlenecks are eliminated, but network connection overhead increases
Solution Approach 1:
The patent applies preliminary action by having edge servers record expiration times at the moment of upload. This pre-established expiration information is stored locally, allowing edge servers to perform deletions without needing to query the central storage or database, thereby eliminating ongoing network overhead for deletion operations.
Data Source
AI summary
The solution distributes the management of stream segments from a central storage cluster to different edge servers that upload stream segments to and receive stream segments from the central storage cluster. Each edge server tracks the stream segments it has uploaded to the central storage cluster as well as the expiration times for those segments. The tracking is performed without a database using a log file and file system arrangement. First-tier directories are created in the file system for different expiration intervals. Entries under the first-tier directories track individual segments that expire within the expiration interval of the first-tier directory with the file system entries being files or a combination of subdirectories and files. Upon identifying expired stream segments, the edge servers instruct the central storage cluster to delete those stream segments. This removes the management overhead from the central storage cluster and implements the distributed management without a database.