Dynamic Cluster Node Expansion for Upgrade Failover
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual computing systems face challenges during upgrades at disaster recovery sites, where the lack of sufficient CPU cores and memory can hinder the migration of user VMs experiencing failover events, as node configuration freezing prevents adding or removing nodes during upgrades, leading to potential errors and operational disruptions.
Innovation Solution
The method involves calculating the required resources for a disaster condition, adding compute-only nodes or nodes with storage to the cluster before the upgrade, and removing unnecessary nodes after the upgrade is complete, ensuring the cluster can handle failover events even during upgrades.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If nodes are added to the cluster before upgrade to provide sufficient resources for failover events, then the disaster recovery capability is improved, but the device complexity increases due to dynamic node management
Solution Approach 1:
The system performs preliminary actions by adding compute-only nodes to the cluster before the upgrade process begins. This ensures that sufficient resources are available to handle failover events during the upgrade, when the cluster might otherwise lack capacity. The nodes are added in advance, configured, and made available before any upgrade operations commence.
Solution Approach 2:
The system implements dynamic node management by automatically adding nodes before upgrade and removing them after upgrade completion. The node configuration is not frozen during the upgrade process, allowing the cluster to adapt its composition dynamically. This dynamic approach enables the cluster to have enhanced capacity during upgrade while returning to its original state afterward.
2Stability of the object's composition
If node configuration is frozen during upgrade to prevent errors, then system stability is improved, but the adaptability worsens as nodes cannot be added or removed during upgrades
Solution Approach 1:
Instead of freezing the node configuration during upgrade, the system performs the necessary node additions as a preliminary action before the upgrade begins. This allows the cluster to be properly configured and expanded in advance, then maintains stability during the actual upgrade process while having already adapted its composition to meet future failover requirements.
Solution Approach 2:
The upgrade process is segmented into distinct phases: a preparation phase where nodes are added and configured, and an execution phase where the upgrade occurs with stable configuration. This segmentation allows adaptability during preparation while maintaining stability during execution, resolving the contradiction between these two requirements.
3Quantity of substance
If compute-only nodes are added before upgrade to handle failover events, then the resource sufficiency is improved, but the loss of time increases due to node addition and removal processes
Solution Approach 1:
The system performs node addition as a preliminary action before the upgrade, ensuring resources are available in advance. While this does consume time, it prevents potential failures during upgrade that would require even more time to resolve. The node removal after upgrade completes the cycle, but the critical resource availability is established beforehand.
Solution Approach 2:
The system uses compute-only nodes that are temporarily added for the upgrade period and then removed. These nodes serve a specific temporary purpose (providing resources during upgrade) and are discarded afterward. This approach minimizes long-term resource commitments and allows for efficient, time-limited resource expansion without permanent infrastructure changes.
4Productivity
If nodes are removed after upgrade to restore original configuration, then resource efficiency is improved, but the productivity decreases due to extended node removal processes
Solution Approach 1:
The compute-only nodes are designed as temporary, disposable resources that serve a specific short-term purpose during upgrade. After fulfilling their function, they are removed from the cluster. This approach accepts the time cost of removal as a trade-off for maintaining optimal long-term resource efficiency and avoiding permanent infrastructure bloat.
Solution Approach 2:
The system discards the temporary compute-only nodes after they have served their purpose during the upgrade process. This discarding allows the cluster to return to its original, optimized configuration. The nodes are recovered (removed) from the cluster environment, completing the temporary resource expansion cycle and restoring resource efficiency.
Data Source
AI summary
Systems and method for expanding a cluster with nodes before an upgrade so that the cluster can handle a failover event during an upgrade and still continue operating. The system and method includes receiving, at a processor, a signal indicating a cluster is about to commence an upgrade; computing, by the processor, a first number of nodes; adding, by the processor, the first number of nodes from a network to the cluster before the upgrade is performed on the cluster; receiving, at the processor, a signal indicating the cluster finished the upgrade; and removing a second number of nodes that is no greater than the first number of nodes from the cluster after the processor receives the signal indicating the upgrade is complete.