Distributed Storage Management for Heterogeneous Media in High Availability Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current high availability cluster technologies fail to effectively manage heterogeneous storage media, particularly when they differ drastically in cost and performance, leading to challenges in maintaining high availability of popular files across nodes.
Innovation Solution
A distributed storage medium management protocol is implemented in a high availability cluster, where files are written to both high-access-rate and low-access-rate storage media, with monitoring and migration mechanisms to ensure availability and re-balance data based on file popularity and storage capacity, allowing for seamless failover and efficient use of storage resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If files are stored on heterogeneous storage media with different access rates, then storage cost is reduced, but file availability deteriorates when high-access-rate media fails
Solution Approach 1:
The patent segments file storage across multiple storage media types within the same cluster. Popular files are stored on high-access-rate media (e.g., SSDs) while less popular files are stored on low-access-rate media (e.g., HDDs). This segmentation allows the system to optimize for both cost and performance by placing different files on appropriate media types.
Solution Approach 2:
The patent implements dynamic migration of files between different storage media based on access patterns and popularity. When a file becomes popular, it is automatically migrated from low-access-rate media to high-access-rate media. This dynamic adjustment ensures that files maintain high availability while the system overall achieves cost savings through heterogeneous storage utilization.
2Reliability
If files are replicated across multiple nodes for high availability, then file availability is improved, but storage efficiency deteriorates
Solution Approach 1:
The patent applies local quality by providing enhanced storage (high-access-rate media) specifically for popular files that require high availability, while using standard storage for less popular files. This targeted approach ensures that replication and high-performance storage are applied only where necessary, improving overall storage efficiency while maintaining file availability for critical files.
Solution Approach 2:
The patent changes the replication parameter dynamically based on file popularity and access patterns. Popular files are replicated across multiple nodes with high-access-rate media, while less popular files use standard replication strategies. This parameter adjustment optimizes the balance between availability and storage efficiency.
3Speed
If high-access-rate storage media is used for all files, then file access speed is improved, but storage cost increases
Solution Approach 1:
The patent applies partial action by providing high-access-rate storage media only for popular files that require fast access, rather than for all files in the system. This selective approach ensures that files requiring high speed get the necessary performance while the overall system achieves cost savings through heterogeneous storage utilization.
Solution Approach 2:
The patent dynamically adjusts file placement between high-access-rate and low-access-rate media based on changing access patterns. Files that become popular are automatically migrated to high-access-rate media, while files with declining popularity are migrated to cost-effective low-access-rate media. This dynamic approach ensures optimal performance-to-cost ratio.
4Quantity of substance
If data is migrated between storage media, then storage optimization is improved, but system complexity increases
Solution Approach 1:
The patent implements self-service through automated migration mechanisms that monitor file access patterns and automatically migrate files between storage media types. The system uses popularity metrics and access statistics to trigger migrations without manual intervention, reducing the operational complexity while achieving storage optimization.
Solution Approach 2:
The patent employs feedback mechanisms that continuously monitor file access patterns and popularity metrics. This feedback drives automated migration decisions, where files are moved between storage media types based on real-time or near-real-time access statistics. The feedback loop ensures storage optimization while automating the process to manage system complexity.
Data Source
AI summary
A high availability cluster and method of storage medium management in such high availability cluster. A number k of nodes belonging to the cluster, where k≧2 and includes a master node, are provisioned with high-access-rate and low-access-rate storage media. A file is written to the high-access-rate medium of a serving node selected from among k−1 nodes excluding the master node. The file is also written to low-access-rate medium of each of k−2 nodes excluding the master node and the serving node. A distributed storage medium management protocol supervises file migration from the low-access-rate medium of a back-up node to the high-access-rate medium of the back-up node upon disruption of file availability on the serving node. File rebalancing relies on parameters including popularity, write- and read-requests, capacity, processing load or cost.