Storage Cluster Redundancy Using Spot and On-Demand Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for configuring Virtual Storage Software Block (VSS Block) clusters using low-cost spot instances in cloud environments do not consider cluster topology, leading to potential shutdowns and loss of volume accessibility, which increases costs and requires separate storage clusters for development and production environments.
Innovation Solution
A method and system for managing storage clusters that pair high reliability and low reliability nodes, utilizing active and standby pairs across these nodes, and selecting storage clusters based on volume policies and the presence of spot instances to optimize volume placement and reduce costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If spot instances are used to form storage clusters, then cost is reduced, but reliability deteriorates because spot instances can be shut down without notice
Solution Approach 1:
The storage cluster is segmented into multiple node types (spot instances and on-demand instances) with different reliability characteristics. The system segments volumes into different categories (development and production) and places them on appropriate node types, allowing cost optimization for development while maintaining reliability for production.
Solution Approach 2:
Different parts of the storage system have different quality characteristics. Development volumes are placed on cost-effective spot instances, while production volumes are placed on reliable on-demand instances. This local differentiation allows the system to optimize both cost and reliability according to local requirements.
2Reliability
If different storage clusters are used for development and production environments, then reliability is maintained for production, but device complexity increases
Solution Approach 1:
A single storage cluster system handles both development and production workloads by dynamically selecting appropriate node types based on volume policies. The system provides multi-functionality by supporting both cost optimization for development and reliability for production within the same cluster framework, eliminating the need for separate clusters.
3Quantity of substance
If spot instances are used for all volumes, then cost is minimized, but productivity deteriorates due to potential shutdowns and access loss
Solution Approach 1:
The system dynamically selects node types based on volume policies and workload requirements. Development volumes can be dynamically placed on spot instances for cost optimization, while production volumes are dynamically placed on on-demand instances for reliability, allowing the system to adapt to different productivity requirements.
Data Source
AI summary
Management of storage clusters involving high reliability and low reliability nodes, wherein active and standby pairs are chained across the high reliability and low reliability nodes such that each of the high reliability and low reliability nodes involve volumes of the active and standby pairs, including, for receipt of a request for volume creation with a specified volume application, selecting a storage cluster from the storage clusters to create the volume with the specified volume application based on policy of the volume and existence of spot instances in the storage clusters, the determining involving, for the policy of the volume indicative of a development volume, selecting the storage cluster having one or more spot instances from the storage clusters; and for the policy of the volume indicative of a production volume, selecting the storage cluster having no spot instances from the storage clusters.