External Distributed Storage Upgrade Orchestration Across Namespaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of upgrading an external distributed storage layer in a container hosting platform is complicated by the incompatibility between the declarative programming model used by the container hosting platform and the imperative programming model used by the external storage layer, leading to difficulties in communication and orchestration of upgrades.
Innovation Solution
A custom-defined operator and cluster controller are implemented within the container hosting platform to translate and orchestrate upgrade commands between the two different programming models, ensuring non-disruptive upgrades by managing nodes in a protection domain-aware sequence, using a node management custom resource to facilitate communication and track progress.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a container hosting platform uses a declarative programming model to manage containers, then automation and ease of operation are improved, but compatibility and ease of communication with external systems using imperative programming models deteriorate
Solution Approach 1:
An intermediary layer is introduced between the container hosting platform and the external distributed storage layer to translate between the declarative programming model used by the platform and the imperative programming model used by the storage layer. This intermediary enables seamless communication and orchestration without requiring changes to either system, resolving the compatibility issue while preserving the automation benefits of the declarative model.
2Reliability
If upgrades are performed on the external distributed storage layer, then system functionality and reliability are improved, but service continuity and client access may be disrupted
Solution Approach 1:
The system performs preliminary actions by creating snapshots of the storage layer state before upgrades and preparing rollback mechanisms in advance. This allows the upgrade process to proceed while maintaining the ability to restore previous states if issues arise, ensuring service continuity and preventing harmful disruptions to client access.
Solution Approach 2:
The system implements beforehand cushioning by establishing protection domains and defining upgrade sequences that ensure at least one node remains operational during upgrades. This cushioning mechanism prevents complete service disruption by maintaining redundant capacity throughout the upgrade process, allowing continuous client access even while nodes are being upgraded.
3Object-affected harmful factors
If nodes are upgraded sequentially to maintain service continuity, then client access is preserved, but upgrade time and productivity are increased
Solution Approach 1:
The storage layer is segmented into multiple independent nodes organized in protection domains, allowing parallel upgrade operations across different segments. By dividing the system into manageable units that can be upgraded independently, the overall upgrade time is reduced while maintaining service continuity through the coordinated sequencing of node upgrades across segments.
Data Source
AI summary
Techniques are provided for upgrading an external distributed storage layer that provides storage services to containerized applications hosted within a container hosting platform. An operator within the container hosting platform is custom configured to orchestrate, from within the container hosting platform, the upgrade for the external distributed storage layer. Because the external distributed storage layer and the container hosting platform are separate computing environment that utilize different namespaces, semantics, operating states, and/or application programming interfaces, a cluster controller within the container hosting platform is custom configured to reformat/translate commands between the external distributed storage layer and the container hosting platform for performing the upgrade. Because the external distributed storage layer upgrade may be part of an overall upgrade that upgrades the containerized applications hosted within the container hosting platform, the operator and cluster controller provide a single upgrade orchestration point for perform both upgrades in an orchestrated manner.