Heterogeneous Distributed File Systems With Tiered Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional distributed file systems assume homogeneous storage mediums, which can lead to inefficiencies in data access speed and storage utilization, especially when dealing with varying storage technologies and sizes.
Innovation Solution
Implementing a heterogeneous distributed file system that utilizes both fast and slow storage mediums, where active data is stored on faster media like SSDs and inactive data on slower media like HDDs, with redundant copies distributed across nodes to ensure data availability and minimize contention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If homogeneous storage mediums are used in distributed file systems, then system simplicity is maintained, but data access speed and storage utilization efficiency deteriorate
Solution Approach 1:
The patent segments the storage system into multiple tiers with different performance characteristics. Fast storage mediums (e.g., SSDs) form the hot tier for actively accessed data, while slow storage mediums (e.g., HDDs) form the cold tier for inactive data. This segmentation allows the system to optimize data access speed for active data while maintaining overall system simplicity through a structured tiered architecture.
Solution Approach 2:
The patent applies local quality by assigning different storage mediums to different data based on access patterns. Frequently accessed data is placed on fast storage mediums, while infrequently accessed data is placed on slow storage mediums. This localized optimization improves data access speed for active data without requiring the entire system to use expensive fast storage.
2Speed
If fast storage mediums are used for all data, then data access speed is improved, but storage cost and storage requirements worsen
Solution Approach 1:
The patent implements dynamic data placement and tiering mechanisms that automatically move data between fast and slow storage mediums based on access patterns. When data is actively accessed, it is placed on fast storage; when inactive, it is moved to slow storage. This dynamic approach optimizes the use of expensive fast storage capacity while maintaining data access speed for active data.
Solution Approach 2:
The patent changes the storage medium parameter based on data activity status. Active data is stored on fast storage mediums with high access speed but high cost, while inactive data is stored on slow storage mediums with lower access speed but lower cost. This parameter change optimizes both data access speed and storage capacity utilization.
3Reliability
If data is replicated across multiple nodes, then data availability and reliability are improved, but storage capacity requirements worsen
Solution Approach 1:
The patent uses copying to create redundant data copies across multiple nodes in the distributed file system. Each data block is replicated to multiple nodes to ensure data availability and reliability. The tiered storage architecture allows these copies to be distributed across both fast and slow storage mediums, optimizing both reliability and storage capacity utilization.
4Productivity
If tiered storage architecture is implemented, then storage utilization efficiency is improved, but device complexity worsens
Solution Approach 1:
The patent implements a universal tiered storage architecture where each node in the distributed file system can function with both fast and slow storage mediums. The system provides multi-functionality by handling both active and inactive data across different storage tiers, improving storage utilization efficiency while distributing the complexity across multiple nodes rather than concentrating it in a single system.
Data Source
AI summary
According to one general aspect, a system may include a plurality of data nodes. Each data node may include either or both of a first storage medium and a second storage medium that is slower than the first storage medium. Each data node may be configured to store a piece data in either the first storage medium or the second storage medium. The system may be configured to store a plurality of copies of an active piece of data within two or more data nodes. A fast copy of the active piece of data may be stored by a first storage medium of a first data node. One or more slow copies of the active piece of data may be stored by respective second storage mediums of one or more respective other data nodes.