Block Map Caching for Distributed Video Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed storage systems for streaming video data face challenges in efficiently managing storage bandwidth and capacity while maintaining scalability and reducing complexity, particularly in redirecting streaming read requests across multiple video servers.
Innovation Solution
A distributed storage architecture utilizing block map caching and a VFS stackable file system, which enables multiple video servers to cooperatively stream assets from a common pool of storage through a storage area network, employing tiered caching to decouple streaming bandwidth from asset storage bandwidth and optimize resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed storage systems use multiple video servers to stream video data, then streaming bandwidth and capacity are improved, but system complexity and overhead increase
Solution Approach 1:
The patent introduces a file server as an intermediary component that mediates between video servers and storage devices. The file server handles file access, block map caching, and storage redirection, allowing video servers to focus on streaming without managing storage complexity. This intermediary layer resolves the contradiction by centralizing storage management functions while maintaining distributed streaming capability.
Solution Approach 2:
The system segments storage management functions into separate components: the file server handles metadata and block maps, while video servers handle streaming. This segmentation allows each component to specialize in its function, improving overall streaming bandwidth while reducing the complexity burden on individual video servers.
2Speed
If block map caching is implemented across multiple video servers, then storage access efficiency is improved, but memory overhead increases
Solution Approach 1:
The patent merges the block map caching functionality across all video servers into a centralized cache on the file server. Instead of each video server maintaining separate block map caches that consume memory, the system combines these caching resources into a single shared cache. This approach maintains fast storage access efficiency while eliminating redundant memory overhead across multiple servers.
Solution Approach 2:
The file server's block map cache serves as a universal resource for all video servers in the system. Rather than dedicating separate cache memory to each video server, the universal cache on the file server serves multiple video servers simultaneously, improving storage access efficiency for all while minimizing total memory overhead through resource sharing.
3Adaptability or versatility
If tiered caching is used to decouple streaming bandwidth from storage bandwidth, then resource allocation is optimized, but system architecture complexity increases
Solution Approach 1:
The patent segments the storage system into two distinct tiers: a file server tier that manages block map caching and metadata, and a storage device tier that holds actual video data. This segmentation creates a clear architectural boundary that simplifies resource allocation - the file server can independently manage cache resources while storage devices handle raw data. The segmented architecture provides resource allocation flexibility without requiring complex multi-layered systems.
Solution Approach 2:
The file server acts as an intermediary tier between video servers and storage devices, implementing the tiered caching architecture. This intermediary layer provides the flexibility to decouple streaming bandwidth from storage bandwidth by handling block map caching and redirection, while keeping the overall architecture relatively simple through a clear two-tier structure rather than multiple complex layers.
Data Source
AI summary
A distributed storage architecture and tiered caching system are employed in a video-on-demand or streaming media application. An illustrative embodiment of a distributed storage architecture, based on block map caching and virtual file system stackable file system modules, includes a controller, a first computer and a second computer, first and second switches, and a storage device. The first computer includes a local file system and uses this to store asset files in the local file system on the first storage device. The first computer employs a process to create a block map for each asset file, the block map including information concerning boundaries where an asset file is stored on the first storage device.