Heterogeneous Storage Tiering for Faster Cloud Node Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing clusters running on cloud infrastructure face deficiencies in non-volatile storage, leading to capacity and functionality limitations, resulting in prolonged recovery times upon node loss.
Innovation Solution
Implementing a heterogeneous storage tiering regime that utilizes node-local storage as a hot tier and cloud-provided storage as a cold tier, allowing flexible promotion and demotion of data between tiers to optimize performance and recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud-specific bare-metal instance configurations are used, then cloud infrastructure compatibility is improved, but non-volatile storage capacity and functionality are limited
Solution Approach 1:
The storage system is segmented into two distinct layers: cloud-provided network-accessible storage (cold tier) for capacity and cloud-attached local storage (hot tier) for performance. This segmentation allows each layer to fulfill different requirements - the cold tier provides abundant capacity while the hot tier provides high-speed access, resolving the contradiction between storage quantity and cloud compatibility.
2Quantity of substance
If more local storage is provisioned on bare metal nodes, then storage capacity is improved, but cost and complexity of node configuration increase
Solution Approach 1:
The system merges cloud-provided network-accessible storage with cloud-attached local storage into a unified heterogeneous storage tiering regime. This combination allows the system to achieve greater total storage capacity without increasing individual node complexity, as the cloud-managed cold tier handles capacity while the local hot tier maintains simple, fast access.
3Reliability
If data is replicated across multiple nodes for high availability, then reliability is improved, but storage capacity consumption increases
Solution Approach 1:
The system applies local quality by storing replicated data in the cold tier (cloud-provided storage) rather than consuming hot tier capacity. This allows high availability replication to occur without unduly consuming performance tier capacity, as replicas are placed in the appropriate storage layer based on their access requirements.
4Quantity of substance
If cloud-provided network-accessible storage is used for replicas, then storage capacity is improved, but access speed decreases
Solution Approach 1:
The system adds a temporal dimension to storage access by implementing a tiered architecture where data moves between cold and hot tiers based on access patterns. Frequently accessed data is promoted to the hot tier for fast access, while less frequently accessed replicated data remains in the cold tier, resolving the speed-capacity tradeoff through time-based differentiation.
Data Source
AI summary
Methods, systems, and computer program products for rebuilding data of a cloud-attached computing cluster. In operation, a computing cluster having at least two nodes is configured to host virtualized entities that are configured to access a common storage access address space. The nodes have respective node-local non-volatile storage devices. The node-local non-volatile storage devices are configured into a first storage tier that stores node-local data. Additional cloud-provided storage is attached to the nodes such that the additional cloud-provided storage forms a second storage tier that stores replicas of the node-local data. In the event of a loss of node functionality, node recovery is accomplished by attaching the additional cloud-provided storage to a replacement node and then rebuilding, on the replacement node, rebuilt first storage tier data based on promotion of certain replicas of the node-local data that are stored in the second storage tier of the additional cloud-provided storage.