Rolling Upgrade of Distributed Software Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed software upgrades typically require all nodes to be either at the old or new version, leading to unavailability and extended downtime, as existing systems cannot function with multiple versions simultaneously, making it unacceptable for organizations that cannot afford prolonged software unavailability.
Innovation Solution
A system and method for rolling upgrades allow multiple nodes to operate different versions of distributed software, with a specified acting version ensuring compatibility, enabling automatic completion of the upgrade process over any duration, and allowing nodes to join or operate based on version compatibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all nodes are upgraded simultaneously to the new version, then the software version is unified, but the software becomes unavailable during the upgrade process
Solution Approach 1:
The system divides the node population into segments that can operate at different software versions simultaneously. Each node is independently upgradeable without requiring coordinated shutdown of the entire system, allowing the upgrade process to be segmented across multiple nodes over time rather than requiring a monolithic simultaneous upgrade.
Solution Approach 2:
The system performs preliminary actions by having nodes check their software version against the acting version before joining or rejoining the cluster. This preliminary version verification allows nodes to be upgraded individually while maintaining system functionality, as the system is prepared to handle mixed-version scenarios through version checking mechanisms.
2Reliability
If nodes operate with different software versions simultaneously, then software availability is maintained, but version compatibility issues arise
Solution Approach 1:
The acting version serves as an intermediary concept that mediates between nodes running different software versions. The acting version acts as a common reference point that all nodes must respect when joining or rejoining the cluster, simplifying version compatibility management by providing a unified interface rather than requiring direct pairwise compatibility checks between all node versions.
Solution Approach 2:
The system implements feedback mechanisms where nodes continuously check their software version against the acting version before joining or rejoining the cluster. This feedback loop ensures version compatibility is maintained dynamically, with nodes receiving information about the current acting version and adjusting their behavior accordingly, thereby managing version complexity through automated verification.
3Reliability
If nodes are upgraded one at a time, then software availability is maintained, but the upgrade process extends over a longer period
Solution Approach 1:
The system maintains continuity of useful action by ensuring that upgraded nodes can immediately join or rejoin the cluster and continue providing services. Rather than taking nodes offline for extended periods, the system allows continuous operation with mixed versions, where nodes can be upgraded, validated, and reintegrated without interruption to overall system functionality, thereby reducing the effective duration of the upgrade process.
Data Source
AI summary
A system and method for facilitating an automatic rolling upgrade of distributed software. In a set of computing devices (e.g., a cluster), each node executes the distributed program, but different nodes may operate any number of different versions of the program. During a rolling upgrade of the software, the nodes all operate according to a specified acting version of the software. A node attempting the join the cluster may be required to have a software version at least equal to the acting version. When all nodes have been upgraded to a version newer than the current acting version, the acting version is reset to the lowest software version installed on a node. The rolling upgrade completes automatically when all nodes run at a specified software version (e.g., the most recent) or have the same version installed.