Hierarchical Storage System for Cost Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hyperscale storage systems face high costs due to the use of expensive high-speed solid state drives (SSDs) for ensuring high availability and data reliability across multiple available zones, which increases the total cost of ownership as the infrastructure scales out.
Innovation Solution
Implementing a hierarchical storage system that uses low-cost SSDs and HDDs in a layered structure, where each available zone includes one low-cost SSD and two low-cost HDDs, allowing for efficient data replication and asynchronous writing, thereby reducing costs while maintaining high availability and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If high-speed SSDs are used for all replicas in multiple available zones to ensure high availability and data reliability, then data reliability and access speed are improved, but the total cost of ownership increases significantly
Solution Approach 1:
The storage system is segmented into multiple storage tiers (fast SSD tier and cost-effective HDD tier) with different performance characteristics. The system divides the storage function across these tiers, allowing critical data to reside on SSDs while less critical data resides on HDDs, thereby resolving the contradiction between reliability and cost.
Solution Approach 2:
Different storage devices are assigned different qualities based on local requirements. Fast SSDs are deployed where high access speed is locally required (for frequently accessed data), while cost-effective HDDs are used where lower cost is prioritized (for less frequently accessed data), achieving optimal balance between performance and cost at each location.
2Speed
If all replicas are stored on high-speed SSDs to meet fast access requirements, then access speed is improved, but the cost efficiency of the storage system deteriorates
Solution Approach 1:
Instead of applying high-speed SSD storage to all replicas (excessive action), the system applies SSD storage only to the extent necessary for meeting access requirements (partial action). This allows the system to achieve acceptable access speeds for critical data while using more cost-effective storage for less critical data, improving overall cost efficiency.
3Reliability
If multiple available zones with multiple replicas are implemented to ensure high availability, then system reliability is improved, but the total cost of ownership increases as the infrastructure scales out
Solution Approach 1:
The system segments storage across multiple availability zones with a mix of storage tiers. Each zone maintains redundancy but uses a combination of SSD and HDD storage, allowing the system to scale out while controlling costs through the heterogeneous storage architecture.
Solution Approach 2:
The storage system uses a composite architecture combining different storage technologies (SSD and HDD) in a unified system. This composite approach allows the system to achieve high availability through redundancy while managing costs by leveraging the cost-effectiveness of HDDs for non-critical data storage.
Data Source
AI summary
One embodiment facilitates data placement in a storage device. During operation, the system receives, from a host, a request to read data. The system determines that the data is not available in a read cache. The system issues the read request to a solid state drive and a first hard disk drive. In response to unsuccessfully reading the requested data from the solid state drive and successfully reading the requested data from the first hard disk drive, the system sends the requested data to the host. In response to unsuccessfully reading the requested data from both the solid state drive and the first hard disk drive: the system issues the read request to a second hard disk drive; and the system sends the requested data to the host.