Cloud Pod Upgrade Automation via Feedback Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Monitoring and managing the performance of large cloud systems with replicated resource units is challenging due to their size and replication structure, leading to difficulties in ensuring proper functioning and recent upgrades across all units.
Innovation Solution
A method using a cloud orchestration platform to identify a data set for a client, access a pod generated via the cloud orchestration platform, and detect input triggers for upgrades. The method interrupts the default pod-replacement protocol by transmitting a custom script that polls microservices for status information and determines conditions for incremental advancement of upgrades.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a default pod-replacement protocol is used for upgrades, then the upgrade process is automated, but the system stability and performance may be compromised due to uncontrolled replacement of pods and replicas
Solution Approach 1:
The custom script implements feedback mechanisms by polling the microservice for status information of pods and replicas during the upgrade process. The container orchestration platform uses this feedback to determine whether conditions for iteration advancement are satisfied, enabling controlled replacement based on real-time system state monitoring
Solution Approach 2:
The custom script performs preliminary actions by checking conditions for iteration advancement before each pod replacement. The container orchestration platform verifies that upgrade conditions are met (such as waiting for replica completion before replacing master pod) to ensure system stability is maintained during the automated upgrade process
2Productivity
If rapid pod replacement is performed during upgrades, then the upgrade speed increases, but the system performance and stability may deteriorate due to uncoordinated replacements
Solution Approach 1:
The upgrade process becomes dynamic rather than static. The container orchestration platform adjusts the replacement protocol based on real-time conditions by polling microservice status information. The system can pause, accelerate, or modify the replacement sequence based on whether upgrade conditions are satisfied, optimizing both speed and stability
Solution Approach 2:
The upgrade process is segmented into controlled iterations rather than a single bulk operation. Each pod or replica is replaced in discrete steps with condition checks between iterations. The custom script divides the upgrade into manageable segments (master pod replacement, replica replacement) that can be coordinated for optimal system performance
3Device complexity
If all pods and replicas are upgraded simultaneously, then the upgrade process is simplified, but the system cannot maintain proper functioning during the transition
Solution Approach 1:
The upgrade process is segmented by role (master pod vs. replicas) and executed in a coordinated sequence. The custom script implements different replacement logic for master pods and replicas, ensuring that replicas are replaced after master pod completion. This segmentation maintains system functionality while keeping the process manageable
Solution Approach 2:
The custom script performs preliminary actions by establishing the correct upgrade sequence before execution. It determines that master pod must be replaced before replicas based on dependency relationships. This preliminary planning simplifies the overall process while ensuring system functionality is maintained during transition
4Reliability
If continuous monitoring of all pods and replicas is performed, then the system can ensure proper functioning, but the monitoring complexity and overhead increase
Solution Approach 1:
The custom script serves multiple functions: it monitors system status, determines upgrade conditions, controls replacement timing, and coordinates the upgrade process. By consolidating these functions into a single monitoring and control mechanism, the system achieves comprehensive monitoring without proportionally increasing complexity
Solution Approach 2:
The microservice performs self-monitoring by providing its own status information to the container orchestration platform through polling. The system monitors its own state (replica completion, master pod status) without requiring external complex monitoring infrastructure, reducing overall system complexity
Data Source
AI summary
A pod hosted on a node is generated via the cloud orchestration platform. The pod is to be assigned to managing the one or more task processes corresponding to the data set. The pod is designated as a master pod. E ach of one or more worker pods replicates data from the master pod. Each of the one or more worker pods is hosted on another node. An input that triggers an upgrade for the corresponding virtual machines is detected. Execution of a custom script is initiated, which triggers: polling the microservice for status information of the pod and the set of replicas; and determining whether a condition for iteration advancement for the upgrade is satisfied based on the status upgrade. An incremental advancement of the upgrade to a next virtual machine is initiated upon determining that the condition for iteration advancement for the upgrade is satisfied.