Client-Side Cache for Deduplication Backup Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deduplication backup systems face challenges in improving backup speed and reducing network traffic, as they require efficient recognition of duplicate information and management of large datasets, which can be resource-intensive and inefficient.
Innovation Solution
The implementation of a process that divides protected data into segments, generates fingerprints, groups them into smaller sets, and uses a client-side cache to identify and perform virtual copies of duplicate data, reducing the need for network transfers and processing load by consolidating multiple fingerprints into single virtual copy commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deduplication backup systems identify and transfer only unique data segments, then network traffic is reduced and backup efficiency is improved, but the system requires significant memory resources to store fingerprints and CPU resources to process deduplication operations
Solution Approach 1:
The patent divides protected data into segments and generates fingerprints for each segment. By segmenting the data processing task, the system can manage fingerprints in a more efficient manner, storing only unique segment fingerprints in memory rather than entire data sets, thus reducing memory resource requirements while maintaining deduplication effectiveness.
Solution Approach 2:
The patent extracts only the essential identifying characteristic (fingerprint) of each data segment and stores it in memory, rather than storing the actual data segments. This extraction approach allows the system to track unique data with minimal memory overhead, resolving the contradiction between backup efficiency and memory resource consumption.
2Productivity
If deduplication backup systems identify and transfer only unique data segments, then network traffic is reduced and backup efficiency is improved, but the system requires significant CPU resources to process deduplication operations
Solution Approach 1:
The patent uses cryptographic hashes (fingerprints) as copies or representations of the actual data segments. By processing and comparing these lightweight fingerprint copies rather than the full data segments, the system dramatically reduces CPU resource requirements for deduplication operations while maintaining the ability to identify unique data for efficient backup.
3Reliability
If traditional deduplication systems store all fingerprints in memory, then duplicate recognition is comprehensive, but memory consumption increases significantly
Solution Approach 1:
The patent segments data into smaller units and creates fingerprints for each segment. By segmenting the fingerprint storage requirement, the system can manage memory usage more effectively, storing only the essential fingerprint data rather than complete data representations, thus maintaining comprehensive duplicate recognition with reduced memory consumption.
Solution Approach 2:
The patent transforms the data representation from storing actual data segments to storing cryptographic hash fingerprints. This parameter change in how data is represented and stored enables the system to achieve comprehensive duplicate recognition with significantly reduced memory consumption, as fingerprints are much smaller than the original data segments.
Data Source
AI summary
Protected data is divided into a plurality of segments. For each segment in the plurality of segments, a fingerprint is generated. The plurality of fingerprints is grouped into group(s) of fingerprints and for each group of fingerprints, a group fingerprint is generated. It is determined if the group fingerprint being processed is stored in a cache and if so, the group fingerprint being processed is flagged as one for which a virtual copy will be performed.