Smart Prefetching in Multi-Cloud Snapshots for Faster Application Restore
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data failover techniques fail to meet latency requirements for disaster recovery due to high restore times, especially for latency-sensitive applications.
Innovation Solution
A smart prefetching technique that utilizes metadata and data from a multi-cloud snapshot technology (MST) service to proactively fetch object metadata and data before actual read requests, reducing latency and improving throughput by predicting data access patterns and heuristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated to multiple datacenters for failover, then reliability is improved, but restore time increases causing high latency
Solution Approach 1:
The system performs preliminary actions by prefetching metadata and data objects before actual read requests occur. The prefetcher proactively retrieves data from the object store based on predicted access patterns, so that when read requests arrive, the data is already cached in memory, dramatically reducing restore time while maintaining failover reliability
Solution Approach 2:
The system segments the data retrieval process into separate phases: metadata prefetching and data object prefetching. By segmenting these operations and handling them independently with different strategies, the system optimizes both the speed of metadata access and data access without interfering with each other, thereby reducing overall restore time
2Device complexity
If data is fetched on-demand during restore operations, then device complexity is reduced, but read request latency increases
Solution Approach 1:
The prefetcher performs preliminary data retrieval actions based on predicted access patterns. By anticipating which data objects will be needed and fetching them in advance, the system eliminates the latency associated with on-demand data retrieval while adding minimal complexity through automated prediction and caching mechanisms
Solution Approach 2:
The system uses feedback from actual read requests to refine future prefetching decisions. By monitoring which objects are accessed and adjusting the prefetcher's predictions accordingly, the system continuously optimizes its performance without requiring complex manual configuration, balancing simplicity with performance
Data Source
AI summary
A smart prefetching of operations technique prefetches metadata and/or data of objects stored in a multi-cloud snapshot technology (MST) service associated with an object store. One or more data objects (e.g., an application) at a primary site may be designated for backup or failover to a secondary site, e.g., in the event of failure of the primary site. Smart prefetching logic of the MST is configured to prefetch the object metadata and/or object data from the object store to serve subsequent requests without accessing the object store. To that end, when storing data to the object store as one or more objects, MST maintains specific metadata along with the data of the objects. The technique utilizes the specific metadata to prefetch the object metadata and/or object data before receiving actual read requests for that data to improve the read latency and throughput when restoring the application as soon as possible.