Backup Chunk Caching via Reference Data Profiling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data backups and restores over bandwidth-constrained networks are inefficient due to the need to transmit large amounts of data, leading to long backup and restore times.
Innovation Solution
Profiling and chunking backup data to identify commonalities among backups, creating a base backup reference set that allows for the transmission of identifiers instead of data contents, reducing network bandwidth usage by storing a copy locally and remotely, and prepopulating new clients with reference data for efficient backups and restores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is backed up to remote storage via network, then data backup function is achieved, but backup time becomes long due to bandwidth constraint
Solution Approach 1:
The patent segments backup data into chunks and identifies common chunks across multiple backups. By dividing data into manageable segments and recognizing repeated segments, the system transmits only unique chunk identifiers rather than entire data sets, dramatically reducing network bandwidth consumption and backup time while maintaining complete data backup capability.
Solution Approach 2:
The patent creates a base backup reference set containing common data chunks from multiple backups. Instead of transmitting actual data contents repeatedly, the system copies only the unique portions and references common portions through identifiers in the base set, reducing network transmission requirements while ensuring complete data restoration capability.
2Reliability
If data is restored from remote storage via network, then data restoration function is achieved, but restore time becomes long due to bandwidth constraint
Solution Approach 1:
The patent segments restored data into chunks and matches them against the base backup reference set. By identifying which chunks exist in the base set and which need to be retrieved from remote storage, the system minimizes network data transfer to only the unique portions, dramatically reducing restoration time while ensuring complete data recovery.
Solution Approach 2:
The patent pre-creates a base backup reference set containing common data chunks before restoration operations. This preliminary action enables the restoration process to quickly identify which data can be obtained locally versus which must be retrieved remotely, optimizing restore performance by avoiding unnecessary network transfers of common data.
3Loss of energy
If base backup reference set is maintained, then network bandwidth usage is reduced, but storage requirements increase due to local and remote copies
Solution Approach 1:
The base backup reference set serves multiple functions: it acts as a local cache for common data during restoration, provides a reference for identifying unique data chunks during backup, and enables efficient data retrieval without requiring full remote storage access for every operation. This multi-functionality justifies the storage investment by delivering substantial network bandwidth savings across multiple operations.
Data Source
AI summary
Contents of a plurality of backups that share a common characteristic are profiled. A portion of the plurality of backups is selected as a base backup reference data to be distributed. A first copy of the base backup reference data is stored at a storage of a backup server. A second copy of the base backup reference data is provided for storage at a storage of a client that shares the common characteristic. The client is located remotely from the backup server.