Online Backup System with Global Two Staged Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online backup systems face challenges with global deduplication, as they often require multiple storage nodes, leading to bottlenecks and inefficiencies, especially when handling large numbers of files, and lack the ability to scale to store unlimited data without performance degradation.
Innovation Solution
A distributed hash table-based online backup system that uses a front-end and back-end arrangement with a communication module, deduplication module, and encryption module to interface with users, authenticate data, and store data in a distributed manner across multiple nodes, allowing for global block-level deduplication and replication for redundancy and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional backup systems use multiple storage nodes with indexing databases for global deduplication, then deduplication capability is improved, but system complexity and performance bottlenecks increase
Solution Approach 1:
The patent extracts the indexing database from the traditional backup system architecture and replaces it with a distributed hash table (DHT) mechanism. This removes the centralized indexing component that causes bottlenecks and complexity, while maintaining global deduplication capability through peer-to-peer hash-based identification and storage of data blocks across multiple nodes.
Solution Approach 2:
The patent introduces a distributed hash table as an intermediary mechanism between the backup client and storage nodes. The DHT acts as a mediator that enables efficient lookup and identification of duplicate blocks across the distributed system without requiring a centralized database, thus reducing system complexity while maintaining deduplication functionality.
2Quantity of substance
If conventional systems store data on clustered file systems to handle large numbers of files, then storage capacity is improved, but cost and performance bottlenecks increase
Solution Approach 1:
The patent segments the storage system into independent distributed nodes that can individually handle data blocks. Instead of relying on a single clustered file system, the system divides storage across multiple nodes, each managing its own data blocks. This segmentation enables unlimited storage capacity while avoiding the complexity and performance bottlenecks of traditional clustered file systems.
3Quantity of substance
If virtual tape libraries are added to increase storage capacity, then data storage capability is improved, but block level deduplication advantages are lost
Solution Approach 1:
The patent implements a dynamic distributed storage architecture where nodes can be dynamically added or removed without affecting the deduplication process. The distributed hash table dynamically tracks data blocks across the network, allowing the system to maintain block-level deduplication efficiency even as storage capacity expands through dynamic node addition, unlike static virtual tape library configurations.
4Quantity of substance
If conventional backup systems handle hundreds of millions of files with multiple versions, then data protection coverage is improved, but system performance degrades
Solution Approach 1:
The patent changes the fundamental parameter of data identification from file-based to block-based hashing. By using cryptographic hashes of individual blocks rather than tracking entire files, the system achieves O(1) lookup time for deduplication regardless of the number of files. This parameter change enables the system to handle hundreds of millions of files with multiple versions while maintaining constant performance levels.
Data Source
AI summary
An encryption for a distributed global online backup system with global two-stage deduplication in the absence of an indexing database where data blocks are encrypted using their SHA-1 signatures as encryption keys.