On-Demand Serverless Disaster Recovery With Progressive Restore
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing disaster recovery techniques consume computing resources by maintaining secondary nodes during normal operation and result in prolonged application downtime due to the inability to access data until the entire restore process is complete.
Innovation Solution
Implement on-demand serverless disaster recovery by storing snapshots in a remote object store and using a cloud block map to provide clients with access to both already restored and not yet restored data during the restore process, while conserving computing resources by creating secondary nodes only when needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If secondary nodes are maintained during normal operation for disaster recovery, then reliability is improved, but computing resources are wasted
Solution Approach 1:
The system dynamically creates secondary nodes only when needed (upon failure detection) rather than maintaining them continuously. The secondary node is instantiated on-demand from the remote object store snapshot and destroyed after use, transforming the static resource allocation into a dynamic, event-driven model that eliminates waste during normal operation.
Solution Approach 2:
Instead of maintaining a live secondary node with full computing resources, the system creates a snapshot copy of the primary node's state in a remote object store. This snapshot serves as the basis for reconstructing the secondary node only when failure occurs, reducing resource consumption while preserving recovery capability.
2Reliability
If traditional restore process is used, then data consistency is ensured, but application downtime is prolonged
Solution Approach 1:
The restore process is segmented into two independent paths: already restored data is served from local storage with fast access, while not yet restored data is fetched on-demand from the remote object store. This segmentation allows simultaneous data recovery and application access, eliminating the all-or-nothing restore approach that causes prolonged downtime.
Solution Approach 2:
A cloud block map is introduced as an intermediary layer between the client and the storage systems. The cloud block map tracks which blocks are restored and which need to be fetched from the remote object store, enabling intelligent routing of read requests during the restore process and allowing applications to access data progressively without waiting for complete restoration.
3Quantity of substance
If snapshots are stored in remote object store, then storage cost is reduced, but access speed decreases
Solution Approach 1:
Data access is segmented into two paths based on restoration status: already restored blocks are accessed from local storage (fast path), while unrestored blocks are fetched from the remote object store (network path). This segmentation allows the system to leverage both the speed of local storage and the capacity of remote storage without compromise.
Solution Approach 2:
The system performs preliminary restoration of frequently accessed or critical data blocks to local storage before they are needed by applications. This advance preparation ensures that when applications access this data, it is already available locally at high speed, while less critical or larger datasets can remain in the remote object store.
Data Source
AI summary
Techniques are provided for on-demand serverless disaster recovery. A primary node may host a primary volume. Snapshots of the primary volume may be backed up to an object store. In response to failure, a secondary node and/or an on-demand volume may be created on-demand. The secondary node may provide clients with failover access to the on-demand volume while a restore process restores a snapshot of the primary volume to the on-demand volume. In some embodiments, there was no secondary node and/or on-demand volume while the primary node was operational. This conserves computing resources that would be wasted by otherwise hosting the secondary node and/or on-demand volume while clients were able to access the primary volume through the primary node. Modifications directed to the on-demand volume are incrementally backed up to the object store for subsequently restoring the primary volume after recovery.