Distributed Local Backup via Peer-to-Peer Uploads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized backup and recovery solutions in large organizations face inefficiencies during initial data upload, leading to server resource overload, network congestion, high storage costs, and slow network links, necessitating a more efficient method for backing up client devices.
Innovation Solution
The implementation of a local distributed backup (LDP) system, where client devices calculate a manifest of their files, send it to a central server for deduplication analysis, and then upload missing files to peer devices with sufficient storage capacity, reducing server load and storage needs, while allowing for flexible centralized or distributed data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all client devices upload data to central storage location simultaneously, then backup coverage is achieved, but server resources and network bandwidth are vastly utilized
Solution Approach 1:
The patent segments the centralized backup architecture into distributed peer-to-peer backup units. Each client device is divided into backup roles (source, destination, or both), creating autonomous backup segments that operate independently rather than concentrating all backup operations at a central server.
Solution Approach 2:
The patent extracts the backup function from the central server and distributes it to peer client devices. By taking out the backup capability from the centralized location and placing it at the network edge (client devices), the system eliminates the bottleneck of central server resources during initial backup operations.
2Reliability
If all client devices upload data to central storage location simultaneously, then backup coverage is achieved, but network connections become highly congested
Solution Approach 1:
The patent segments network traffic by creating multiple distributed backup paths instead of a single centralized path. Each peer-to-peer backup creates an independent network segment, distributing traffic load across numerous small connections rather than overwhelming a single central connection.
Solution Approach 2:
The patent changes the network topology from a centralized star pattern (all clients to one server) to a distributed mesh pattern (clients to multiple peers). This dimensional change in network architecture allows parallel backup operations without congesting a single network bottleneck.
3Reliability
If central storage location replicates all client data, then data recovery is enabled, but storage requirements in data-center become costly
Solution Approach 1:
The patent implements self-service backup where client devices automatically identify and backup their own data to peer devices without requiring centralized storage allocation. Each device serves its own backup needs by utilizing storage capacity of other peer devices in the network.
Solution Approach 2:
The patent makes storage capacity universal across the peer network. Storage space on any peer device becomes available to any other peer device, creating a shared pool of storage resources that eliminates the need for dedicated centralized storage for each client.
4Reliability
If central storage location stores all unique files, then data availability is ensured, but I/O operations on server increase
Solution Approach 1:
The patent segments the I/O workload by distributing file storage and access operations across multiple peer devices instead of concentrating all I/O operations at the central server. Each peer device handles its own I/O operations locally, eliminating server I/O bottlenecks.
Data Source
AI summary
Local distributed backup techniques are described, where each client device calculates a manifest of its files and sends the manifest to a central server. The server performs deduplication based on the received manifest to determine which files are missing and need to be backed up, and notifies the client of those files. At this point, instead of uploading the missing files to the central server, the files are instead uploaded to another peer client device that has sufficient disk space and resides in the network vicinity of the original client device. The upload may be done to multiple peers for redundancy. Upon backup completion, the client state may be left as-is, in which case the client periodically uploads incremental updates to keep any changes synchronized with the backup peer. Alternatively, the client can be centralized to the central server, but at a later time when more resources are available.