Database-less Backup System Using Content-Based Addressing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional backup systems face limitations in scaling to store multiple petabytes with redundancy due to resource-intensive database operations and inefficient bandwidth usage, particularly in cloud and box-to-box backup operations.
Innovation Solution
The system eliminates duplicate storage by using atomic file operations to replicate data across multiple servers, leveraging cryptographic hashes to identify unique file characteristics and manage shard distribution without database reliance, allowing for efficient storage and replication of large data quantities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If database technology is used for tracking archives, ownership, and status of shards, then data management capability is improved, but resource requirements and system complexity increase significantly
Solution Approach 1:
The patent extracts the database dependency from the backup system by implementing a database-less architecture. Instead of using databases to track shards, the system uses content-based addressing where shard locations and metadata are derived directly from cryptographic hashes of the data itself, eliminating the need for complex database operations while maintaining full tracking capability
Solution Approach 2:
The system replaces database records with cryptographic copies. Rather than storing metadata about shards in database tables, the patent uses cryptographic hashes and digital signatures that inherently contain all necessary identification and verification information, making separate database tracking unnecessary
2Adaptability or versatility
If database operations are used for shard tracking, then data organization is improved, but instruction executions and operational steps increase
Solution Approach 1:
The patent removes database query operations entirely from the shard tracking process. By embedding all necessary identification information in cryptographic hashes and using content-based addressing, the system eliminates multiple database read/write operations that would otherwise be required to track and locate shards
Solution Approach 2:
The system replaces mechanical database operations with cryptographic verification. Instead of querying database tables to verify shard status and location, the patent uses cryptographic signatures and hash comparisons that can be performed with far fewer computational steps
3Quantity of substance
If conventional backup systems are used for petabyte-scale storage, then basic backup functionality is provided, but bandwidth efficiency deteriorates
Solution Approach 1:
The patent implements self-organizing backup where the system automatically identifies and eliminates duplicates through content-based addressing. By using cryptographic hashes of file contents as shard identifiers, the system inherently recognizes duplicate data across the network without requiring centralized coordination, reducing redundant bandwidth consumption
Solution Approach 2:
The system changes the fundamental parameter used for data identification from metadata-based to content-based. By using cryptographic hashes of actual file contents rather than file names or attributes, the system can efficiently detect and eliminate duplicate shards across the distributed network, significantly reducing bandwidth usage for petabyte-scale operations
Data Source
AI summary
An apparatus to scale for multiple petabyte backup in redundant locations. Workload is automatically shared among many servers by a characteristic derived from the content itself. Duplicate storage is eliminated by checking for the existence or absence of simple files and appending client identification to files shared among multiple subscribers. Replication depends on simple atomic file operations rather than use of tables or databases. An efficient storage method for much larger quantities of data than conventional services.