Deferred Server Recovery in Cloud Datacenters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing datacenters face significant downtime and data loss due to server reboots and failures, which impact virtual machine availability and user experience, as existing recovery methods often result in immediate server restarts affecting multiple virtual machines.
Innovation Solution
Implementing a controller that monitors and detects deferrable hardware and software faults, allowing for scheduled recovery operations, such as delayed reboots, to minimize downtime and data loss by designating affected servers as unavailable for new virtual machine allocations until recovery is necessary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If immediate server recovery (reboot) is performed upon detecting faults, then system reliability is improved, but virtual machine availability deteriorates due to downtime
Solution Approach 1:
The system performs preliminary classification of faults into deferrable and non-deferrable categories before executing recovery actions. The controller analyzes fault characteristics and determines whether immediate recovery is necessary or if the fault can be deferred to a later time, thereby avoiding unnecessary immediate reboots that would cause downtime
Solution Approach 2:
The recovery timing is made dynamic rather than static. Instead of always performing immediate recovery, the system adapts the recovery timing based on fault classification - some faults trigger immediate recovery while others are deferred to scheduled times, optimizing the balance between reliability and availability
2Reliability
If server reboot is performed to correct software faults, then system reliability is improved, but loss of time increases due to downtime
Solution Approach 1:
The system performs preliminary classification of faults into deferrable and non-deferrable categories before executing recovery actions. The controller analyzes fault characteristics and determines whether immediate recovery is necessary or if the fault can be deferred to a later time, thereby avoiding unnecessary immediate reboots that would cause downtime
Solution Approach 2:
The system implements periodic scheduled recovery operations for deferrable faults instead of immediate recovery. The controller schedules recovery at optimal times when impact on virtual machines is minimized, converting continuous availability loss into controlled, predictable periodic downtime
3Productivity
If server is designated as unavailable for new virtual machine allocations, then virtual machine availability is improved, but device complexity increases
Solution Approach 1:
The controller segments fault handling into distinct categories (deferrable vs. non-deferrable) and implements separate handling paths for each. This segmentation allows the system to manage complexity by treating different fault types differently, rather than using a single complex recovery mechanism for all faults
Data Source
AI summary
Various techniques for deferred server recovery are disclosed herein. In one embodiment, a method includes receiving a notification of a fault from a host in the computing system. The host is performing one or more computing tasks for one or more users. The method can then include determining whether recovery of the fault in the received notification is deferrable on the host. In response to determining that the fault in the received notification is deferrable, the method includes setting a time delay to perform a pending recovery operation on the host at a later time and disallowing additional assignment of computing tasks to the host.