Hibernating Computing Cluster Nodes via Cloud Data Tiering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for hibernating and resuming computing clusters face challenges in efficiently determining and migrating 'hot' and 'cold' data, leading to unnecessary resource usage costs and complex data management in cloud computing environments, especially in bare metal clouds where releasing hardware also releases storage.
Innovation Solution
The use of an information lifecycle manager (ILM) to handle data movement between storage tiers, employing built-in capabilities to automatically differentiate between 'hot' and 'cold' data and optimize its migration to cloud or cold storage, thereby reducing costs and simplifying the hibernation and resumption processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If the entire cluster is hibernated to save resources, then resource usage costs are reduced, but data management complexity increases and restoration manual intervention is required
Solution Approach 1:
The system automatically detects hot and cold data, makes tiering decisions, and executes data migration without manual intervention. The hibernation and restoration processes are automated, allowing the system to manage itself and eliminating the need for manual data management during cluster hibernation.
Solution Approach 2:
The system changes the storage tier parameter of data based on access patterns. Hot data remains in high-performance storage while cold data is automatically moved to lower-cost storage tiers, optimizing the balance between performance and cost without increasing management complexity.
2Loss of energy
If cold data is tiered down to lower cost storage, then resource costs are reduced, but data movement complexity increases
Solution Approach 1:
The system automatically identifies cold data and executes tiering operations without manual intervention. The automated detection and migration processes handle the complexity of data movement, making the system self-managing and reducing operational burden.
Solution Approach 2:
The system performs preliminary analysis of data access patterns to identify cold data before tiering occurs. This advance detection and classification prepares the data for automated migration, simplifying the overall data movement process by planning it in advance.
3Reliability
If data is replicated multiple times for high availability, then system reliability is improved, but unnecessary replication and storage waste increases
Solution Approach 1:
The system changes the replication parameter dynamically based on data importance and access patterns. Critical data maintains replication for high availability while less critical cold data has replication reduced or eliminated, optimizing the balance between reliability and storage efficiency.
Data Source
AI summary
Methods, systems and computer program products for hibernating a computing cluster. The present disclosure describes techniques for hibernating and resuming nodes of a computing cluster and entire computing clusters including movement of data and metadata to and from a cloud-tier storage facility (e.g., a cloud disk(s)) in an efficient manner.