Event-Driven Incremental Backup for Cloud Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data backup methods in cloud computing environments are not scalable, costly, and prone to performance issues, security concerns, and provider dependency, especially when using incremental snapshots or kernel modules.
Innovation Solution
Implementing an event-driven mechanism at the device layer to mark dirty buffers in the page cache, accumulate block numbers, and save them as incremental backups to a persistent data store, leveraging eBPF for kernel functionality and user-space interaction, allowing for automatic and fast backups independent of the cloud provider.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full backup is taken each time, then data is captured completely, but time and performance problems become impractical as disk size grows
Solution Approach 1:
The patent segments the backup process into tracking only modified blocks rather than backing up entire disks. The system divides disk data into blocks and uses a bitmap to track which blocks have changed, allowing incremental backups of only necessary segments.
Solution Approach 2:
The patent performs partial action by backing up only the portion of data that has actually changed since the last backup. Instead of performing complete backup actions, the system identifies and backs up only modified blocks, reducing the action to the minimum necessary amount.
2Reliability
If incremental snapshots are taken via provider API, then backup cost is incurred, but frequency and portability are limited
Solution Approach 1:
The patent introduces an intermediary layer (the backup agent and bitmap tracking system) between the disk and the backup destination. This intermediary manages the incremental backup process independently of cloud provider APIs, allowing portability across different providers while maintaining incremental backup capabilities.
3Productivity
If kernel module is used for incremental backup, then system access is complete, but security risks and performance harm increase
Solution Approach 1:
The patent extracts the backup tracking functionality from the kernel space and places it in user space. By using eBPF to attach to kernel functions and retrieve block change information without loading kernel modules, the system maintains backup speed while eliminating the security risks associated with kernel-level code.
4Reliability
If frequent backups are performed, then data protection is improved, but storage cost and time consumption increase
Solution Approach 1:
The patent segments the backup data into individual block-level increments rather than storing complete disk images for each backup. The bitmap mechanism tracks exactly which blocks changed, allowing the system to store only the necessary segments of data for each incremental backup.
Solution Approach 2:
The patent performs partial backups by storing only the blocks that have actually changed since the last backup. This partial action approach reduces storage consumption while maintaining data protection, as the system stores minimal necessary data rather than redundant complete backups.
Data Source
AI summary
Some embodiments may be associated with a cloud computing environment. A device layer may execute a kernel and determine that a write to disk storage is requested by an application executing in the cloud computing environment. A kernel function at the device layer may mark buffers of a page in a page cache that correspond to the requested write as dirty. An event-driven mechanism may send an event to an entry handler in user space, wherein the event has a block number associated with the dirty buffer as a payload. the block numbers received via events may be accumulated in the user space until it is determined that a pre-determined condition associated with the accumulated block numbers is satisfied. At that point, blocks associated with the accumulated block numbers may be saved to a persistent data store as an incremental backup.