Cluster Upgrade Lead Node Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cluster upgrade methods either require rolling upgrades, which are not suitable for major upgrades, or offline upgrades, which result in the cluster being unavailable to service client requests during the upgrade window.
Innovation Solution
The method involves designating a lead node in a cluster that performs the upgrade first, while follower nodes continue to provide services. The lead node migrates applications to follower nodes, updates its operating system, and reboots with the new OS. Once upgraded, the lead node stops services on follower nodes, creates a new cluster, and starts updated applications. Follower nodes are then upgraded and joined to the new cluster, ensuring continuous service availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If offline upgrade method is used, then the cluster can be upgraded to a newer OS version, but the cluster becomes unavailable to service client requests during the upgrade window
Solution Approach 1:
The cluster is segmented into a lead node and follower nodes. The upgrade process is divided into phases: first the lead node is upgraded while follower nodes continue serving clients, then follower nodes are upgraded sequentially. This segmentation allows the upgrade to proceed without taking the entire cluster offline.
Solution Approach 2:
The lead node performs preliminary actions by downloading and extracting the software update to a shared disk before the actual upgrade. This preliminary preparation ensures that when the lead node reboots with the new OS, the update is already ready, reducing the overall upgrade time and ensuring a smooth transition.
2Reliability
If rolling upgrade is used, then service availability is maintained, but it is not suitable for major upgrades
Solution Approach 1:
The upgrade method is made dynamic by allowing the system to adapt its upgrade strategy based on the type of upgrade being performed. For major upgrades, the system uses a lead-node-first approach with sequential follower node upgrades, while for minor upgrades, rolling upgrades can be used. This dynamic approach enables the system to handle both major and minor upgrades effectively.
3Ease of manufacture
If all nodes are upgraded simultaneously, then the upgrade process is simplified, but the cluster becomes unavailable during the upgrade window
Solution Approach 1:
The lead node performs preliminary actions by downloading and extracting the software update to a shared disk before the actual upgrade. This preliminary preparation ensures that when the lead node reboots with the new OS, the update is already ready, reducing the overall upgrade time and ensuring a smooth transition.
Solution Approach 2:
A shared disk acts as an intermediary between the lead node and follower nodes during the upgrade process. The lead node extracts the software update to the shared disk, which then serves as a repository for follower nodes to retrieve the update. This intermediary mechanism simplifies the upgrade process while maintaining service availability.
Data Source
AI summary
An example methodology includes declaring a node in a cluster of multiple nodes as a lead node to perform a cluster upgrade. While the lead node is performing the upgrade operations, the remaining nodes in the cluster continue to provide the cluster services. The upgrade operations on the lead node include migrating applications on the lead node to one of the other nodes in the cluster and performing upgrade of the lead node. In some cases, the upgrade operations include rebooting the lead node with a new OS image and installing new versions of the applications or re-installing the same applications on the lead node. The upgrade operations on the lead node may also include, once the lead node is upgraded, stopping applications and cluster services on the other nodes in the cluster, creating a new cluster that includes the lead node, and starting the applications on the lead node.