Cloud Backup Chunk Indexing and Virtual Volume Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current enterprise backup and disaster recovery systems are inefficient in providing instant restoration of data, especially in cloud storage environments, as they often require restoring entire volumes or files at once, which can be time-consuming and costly, and do not effectively manage data duplication and retrieval from high-latency storage.
Innovation Solution
A method of data backup and restoration that divides data into chunks, maintains an index for efficient retrieval, and uses a virtual volume with a cloud storage interface to cache data as needed, allowing selective and incremental restoration, and leveraging cloud storage for cost-effective and resource-efficient data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If entire volumes are restored at once from cloud storage, then data restoration is completed, but retrieval time and costs increase significantly
Solution Approach 1:
The patent divides backup volumes into smaller chunks and stores them in cloud storage. During restoration, only the specific chunks containing requested data blocks are retrieved and assembled, rather than restoring entire volumes. This segmentation enables selective data recovery, dramatically reducing retrieval time and costs while maintaining complete data availability.
2Speed
If data is cached locally for fast access, then retrieval speed improves, but data duplication and storage costs increase
Solution Approach 1:
The system implements a data cache that pre-loads and stores frequently accessed data chunks locally. Instead of caching entire volumes, only the specific chunks that are likely to be accessed are maintained in the local cache. This partial caching approach provides fast access for hot data while avoiding unnecessary duplication of cold data, optimizing the balance between access speed and storage efficiency.
3Quantity of substance
If cloud storage is used for backup, then storage costs are reduced, but data retrieval latency increases
Solution Approach 1:
The system performs preliminary actions by dividing data into chunks and storing metadata about chunk locations in advance. When data needs to be retrieved, the system uses the pre-stored metadata to quickly locate and fetch only the necessary chunks from cloud storage, rather than searching for data block by block. This preliminary organization of data into manageable chunks with tracked locations significantly reduces retrieval latency while maintaining cost-effective cloud storage usage.
Data Source
AI summary
A system for progressive just-in-time restoration of data from backup media. Backup data may be divided into a plurality of chunks and stored on any kind of media such as a direct attached storage (DAS) disk, object storage, USB drive, network share or tape. An index map is maintained that indicates the location of each of the plurality of chunks in cloud storage, the index map representing contiguous blocks of backup data of a volume. The backup data may be compressed, encrypted, or de-duplicated. The backup data may be located on different media, object stores, or network shares, or at differing geographic locations. To perform a recovery, a virtual LUN or virtual volume is mounted and provided to the operating system and applications of the restored computer. Chunks may be progressively copied from cloud storage to a data cache and restored in response to requests for blocks.