Cloud Filesystem Dynamic Storage Selection Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current file systems fail to aggregate heterogeneous storage types such as NFS, iSCSI, object-based, and cloud storage into a unified, high-performance storage solution, leading to varied latency issues and inefficient data distribution across different storage systems.
Innovation Solution
A cloud-scale file-based filesystem that abstracts multiple storage types into a single mount point, using a database to manage directory structures and UUIDs for unique file identification, and dynamically selects the best storage for write operations based on capacity and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional block-based file systems use round robin or RAID-level storage selection, then data distribution is even across storages, but storage performance varies due to heterogeneous latency characteristics of different storage types
Solution Approach 1:
The patent changes the storage selection parameter from fixed round-robin indexing to dynamic latency-based selection. The file system monitors and measures actual storage latency characteristics, then adapts its storage selection algorithm to choose storages based on current performance metrics rather than predetermined patterns, resolving the contradiction between even distribution and performance consistency
Solution Approach 2:
The system transitions from static storage selection (round robin) to dynamic storage selection that adapts to changing storage conditions. By continuously monitoring storage performance and adjusting selection decisions in real-time, the system maintains both data distribution balance and performance consistency across heterogeneous storages
2Adaptability or versatility
If file-based storage systems aggregate multiple cloud providers and on-premises storages, then storage capacity and versatility increase, but latency varies widely across different storage types
Solution Approach 1:
The patent introduces a storage abstraction layer that acts as an intermediary between the file system and heterogeneous storages. This layer standardizes access paths and implements intelligent routing that accounts for storage-specific latency characteristics, enabling versatile storage aggregation while minimizing latency variations through optimized access patterns
Solution Approach 2:
The system applies storage-specific optimization strategies tailored to each storage type's characteristics. By recognizing and adapting to the unique latency profiles of different storages (cloud vs. on-premises, object vs. file-based), the system optimizes access patterns for each storage location, reducing overall latency while maintaining aggregation versatility
3Ease of operation
If cloud-scale file-based filesystem abstracts multiple storage types into single mount point, then ease of operation improves, but system complexity increases to manage heterogeneous storages
Solution Approach 1:
The patent implements a universal storage abstraction layer that provides a single mount point interface for accessing multiple heterogeneous storages. This layer handles the complexity of storage aggregation, monitoring, and selection internally, while presenting a simplified unified interface to users, thereby improving ease of operation without exposing the underlying system complexity
Data Source
AI summary
A method is provided for a file-based filesystem. The method includes mounting storage devices, periodically ranking the mounted storage devices, exporting a directory in a native filesystem as a share, receiving a first request to make a directory in the share, and, in response to the first request, making the directory in the store by storing information about the directory in a database without making the directory in the native filesystem. The method includes receiving a second request to write a file in the share and, in response to the second request, writing the file in the share by selecting a highest-ranking mounted storage device, storing information about the file in the database, and writing the file with a unique identifier as its file name in a root directory of the selected storage.