Backup System Using Manifest Files for Rapid Change Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current backup systems are time-consuming, labor-intensive, and inadequate for 24/7 networks, often losing file structure during storage, making data retrieval difficult and costly, especially for companies relying heavily on continuous data availability.
Innovation Solution
A distributed metadata store and client software that uses manifest files for local change detection, asynchronous replication correctness validation, and WebDAV for efficient data transfer, enabling rapid identification of changed files, compression, encryption, and parallel processing to minimize network bandwidth and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If backup is performed periodically (once a week or once a day), then labor intensity and time consumption are reduced, but data loss risk increases and backup value decreases
Solution Approach 1:
The patent implements continuous backup by monitoring file system events in real-time and immediately replicating changed data to backup storage, eliminating the gaps inherent in periodic backup schedules. This ensures data protection reliability is maintained without requiring frequent manual intervention, thus resolving the contradiction between backup efficiency and data protection reliability.
Solution Approach 2:
The backup system automatically detects file changes through file system event monitoring and triggers backup operations without requiring administrator intervention. This self-service mechanism enables continuous backup protection while minimizing labor intensity, as the system autonomously manages the backup process based on actual data changes.
2Reliability
If complete data transfer is performed, then backup completeness is ensured, but network bandwidth consumption and time consumption increase
Solution Approach 1:
The patent extracts only the changed portions of files (data deltas) rather than transferring complete files. By identifying and transferring only the specific data blocks that have modified since the last backup, the system ensures backup completeness while dramatically reducing network bandwidth consumption and backup time compared to complete data transfers.
Solution Approach 2:
The system performs partial backup actions by transferring only the necessary changed data portions rather than complete files. This partial action approach maintains backup reliability by capturing all changes while minimizing time consumption and network resource usage.
3Quantity of substance
If data is stored sequentially on media like magnetic tape, then storage cost is reduced, but data retrieval difficulty increases and file structure is lost
Solution Approach 1:
The patent segments data into discrete file-level units with preserved directory structures, allowing individual files and folders to be independently stored and retrieved. This segmentation enables efficient random access to specific data portions without requiring sequential playback, maintaining ease of operation while using cost-effective storage media.
Solution Approach 2:
The system creates complete copies of the file system structure including directories, files, and metadata on the backup media. This copying approach preserves the original file structure and enables straightforward data retrieval operations, making backup data as easily accessible as source data while utilizing inexpensive storage media.
4Productivity
If parallel I/O operations are implemented, then throughput increases and latency is reduced, but system complexity increases
Solution Approach 1:
The patent merges multiple I/O operations into unified backup streams by consolidating changed data from multiple source files into single backup operations. This merging approach enables parallel processing benefits and increased throughput while reducing the overall complexity of managing numerous separate I/O operations.
Data Source
AI summary
A data recovery system includes a plurality of customer computers to be backed-up, each customer computer running a client software to communicate back-up data files; a system management platform coupled to the client software over the Internet, the system management platform receiving inputs from a web user portal to control operations of the client software and the system management platform to back up the customer computer; and two or more data storage silos, each including: a plurality of storage directors communicating with the client software; and a clustered data storage array.