Cloud Backup Restore Endpoint With Lazy-Loaded Object Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based backup systems face challenges in differentiating between data types, managing time-constraints, data size, and cost, and restoring data efficiently, particularly for large incremental backups, which can take substantial time.
Innovation Solution
A system that utilizes a metadata query and lazy loading to access requested objects quickly by inserting metadata in a key/value store like DynamoDB, generating a virtual endpoint for user applications to query, and allowing parallel access to backup data with slower full restoration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full restoration of large incremental backups is performed, then complete data recovery is achieved, but restoration time becomes substantial
Solution Approach 1:
The patent extracts only the metadata necessary for data location and access from the full backup system, storing it separately in a key-value store. This allows rapid access to specific backed-up objects without requiring full restoration of the entire backup, thus reducing restoration time while maintaining data recovery capability for accessed objects
Solution Approach 2:
The system performs preliminary actions by pre-processing and storing metadata about backup objects in a key-value store before actual data access is needed. This metadata includes information about object locations, versions, and access paths, enabling fast retrieval without waiting for full backup restoration to complete
2Speed
If metadata is stored and queried for each backup object, then access speed to specific objects is improved, but system complexity increases
Solution Approach 1:
The patent introduces a metadata layer as an intermediary between the backup storage system and the access interface. This metadata store acts as a mediator that translates access requests into efficient data retrieval operations, improving access speed while managing complexity through abstraction rather than exposing the full system complexity to users
3Ease of manufacture
If cloud-based backup services are used, then storage cost and hardware maintenance are reduced, but access and restoration efficiency deteriorates
Solution Approach 1:
The patent segments the backup system into distinct functional components: metadata management in a key-value store, actual data storage in cloud buckets, and access orchestration logic. This segmentation allows the system to leverage cloud storage ease while implementing efficient access patterns through specialized metadata indexing and query optimization, thus improving data access efficiency without sacrificing cloud storage benefits
Data Source
AI summary
Techniques are disclosed relating to providing fast access to restored data in a cloud storage environment. A computing system stores multiple sets of incremental backup data that reflect changes in data being backed up between backup intervals and metadata that indicates which set of incremental backup data stores a given object. The computing system generates an endpoint for a requesting computing system, where the endpoint supports requests for restored backup data and data responses. In response to a request from the requesting computing system via the endpoint, the computing system queries the metadata based on the request and stores metadata retrieved from the query in a key/value store using object identification information as key data and the retrieved metadata as value data. The computing system provides requested data according to a lazy loading technique, including providing requested data via the endpoint based on the metadata in the key/value store.