Storage Node Upgrade Grouping for Parallel Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Updating multiple storage nodes in a storage service simultaneously without causing data unavailability due to simultaneous node failures during upgrades, especially when the number of nodes is large, is challenging.
Innovation Solution
A system that groups storage nodes into fixed upgrade groups using modulo operations on node metadata to derive group identifiers, allowing parallel updates while ensuring data availability by replicating data across different groups, with a metadata service managing group assignments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple storage nodes are updated simultaneously, then upgrade time is reduced, but data availability deteriorates due to simultaneous node failures
Solution Approach 1:
The system segments storage nodes into distinct upgrade groups based on group identifiers derived from node metadata. This segmentation allows simultaneous updates within each group while maintaining availability through replication across groups, resolving the contradiction between upgrade speed and data availability.
Solution Approach 2:
The system performs preliminary actions by deriving group identifiers from node metadata before updates begin. This pre-organization into groups enables coordinated update scheduling that maintains data availability while achieving parallel updates, thus improving upgrade speed without sacrificing reliability.
2Quantity of substance
If the number of storage nodes increases, then storage capacity improves, but upgrade complexity increases due to coordination requirements
Solution Approach 1:
Storage nodes automatically determine their own group membership by deriving group identifiers from their metadata. This self-service mechanism eliminates the need for centralized coordination complexity when scaling the system, allowing storage capacity to increase without proportionally increasing upgrade coordination complexity.
Solution Approach 2:
The system uses parameter changes in node metadata to derive group identifiers, enabling automatic group assignment as nodes are added or removed. This approach allows the system to scale storage capacity while maintaining simple upgrade coordination through consistent grouping rules.
3Reliability
If updates are performed one node at a time, then data availability is maintained, but upgrade time increases significantly
Solution Approach 1:
By segmenting nodes into upgrade groups, the system enables parallel updates within each group while maintaining data availability through replication across groups. This segmentation approach reduces total upgrade time compared to sequential updates while preserving reliability.
Solution Approach 2:
The system performs partial parallel updates within upgrade groups rather than requiring all nodes to be updated simultaneously or one at a time. This partial action approach optimizes the balance between upgrade speed and data availability by updating multiple nodes in controlled parallel batches.
Data Source
AI summary
Techniques are disclosed relating to upgrade groups. A node of a computer system may access metadata assigned to the node during deployment of the node. The node may be one of a plurality of nodes associated with a service that is implemented by the computer system. The node may perform an operation on the metadata to derive a group identifier for the node and the group identifier may indicate the node's membership in one of a set of groups of nodes managed by the service. The node may then store the group identifier in a location accessible to the service.