Cloud Block Map for Efficient Data Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
During the cloud restore process, deduplicated data is pulled separately via individual API calls from the snapshot and written to a local volume in separate data blocks, leading to loss of storage efficiency and repeated API calls.
Innovation Solution
The implementation of a cloud block map (CBMAP) metafile that maintains a mapping of cloud block numbers (CBNs) to corresponding local volume block numbers, allowing deduplicated blocks to be read only once from the cloud and stored as a single copy on the local volume.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deduplicated data is pulled separately via individual API calls during cloud restore, then data can be recovered from the snapshot, but storage efficiency is lost and repeated API calls occur
Solution Approach 1:
The patent creates a cloud block map (CBMAP) that copies and stores the mapping relationships between cloud block numbers (CBNs) and local volume block numbers. This copy of the mapping information allows the system to retrieve block mappings without repeatedly querying the cloud storage, thereby preserving storage efficiency while enabling complete data recovery.
Solution Approach 2:
The system performs preliminary action by pre-fetching and storing the cloud block map before the actual data restoration process. The CBMAP is created and stored locally, containing all the mapping information needed for restoration. This preliminary preparation eliminates the need for repeated API calls during the restore operation, as the mapping data is already available locally.
2Reliability
If individual API calls are used to retrieve each deduplicated block, then complete data restoration is achieved, but the number of API calls increases and latency increases
Solution Approach 1:
The patent merges multiple separate API call operations into a single batch operation. Instead of making individual API calls for each block mapping query, the system retrieves the entire cloud block map in one operation and stores it locally. This merging of operations dramatically reduces the number of API calls and the associated latency while ensuring complete data restoration.
Solution Approach 2:
The system performs preliminary action by pre-fetching the complete cloud block map before the restoration process begins. This single preliminary API call retrieves all mapping information at once, and subsequent restoration operations use this cached map data without requiring additional API calls, thereby minimizing total latency.
3Reliability
If deduplicated blocks are stored as separate data blocks on local volume, then data can be restored, but duplicate storage occurs and storage efficiency is reduced
Solution Approach 1:
The patent creates a cloud block map that copies the mapping relationships between cloud blocks and local blocks. This copy enables the system to identify and share deduplicated blocks across different files and volumes during restoration, ensuring that identical data blocks are stored only once on the local volume rather than creating duplicates, thus preserving storage efficiency.
Data Source
AI summary
Systems and methods for preserving storage efficiency during restoration of data from the cloud are provided. In one embodiment, a CBMAP is maintained that maps cloud block numbers (CBNs) to respective corresponding block numbers of a volume of a data storage system in which previously restored data has been stored by a previously restored file. By making use of the CBMAP during the restoration process, storage of duplicate file data blocks on the volume may be avoided by sharing with a current file being restored a reference to the corresponding file data block previously stored on the volume and associated with the previously restored file. In addition to preserving storage efficiency, use of the CBMAP facilitates avoidance of repeated GET operations for data associated with CBNs previously retrieved from the cloud and stored to the volume, thereby reducing data access costs as well as latency of the restore operation.