Cloud WORM Backup for File Systems Using Chunk Versioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file-to-object replication techniques lack incremental updates, lose fidelity, and do not support immutability, especially in file system backups to object storage systems.
Innovation Solution
Implement a system that identifies changes in a file system relative to an object chunk, copies changed and unchanged data to a new version of the object chunk, and utilizes object locks in cloud-based object stores to ensure immutability, such as AWS S3, by employing a pseudo-copy-on-write replication method and versioning to maintain data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing file-to-object replication techniques are used, then data can be stored in object storage, but incremental updates are lost and data fidelity deteriorates
Solution Approach 1:
The patent segments data into chunks and tracks versions of each chunk independently. When changes occur, only the affected chunks are updated while maintaining references to previous versions, enabling incremental updates without losing data fidelity. This segmentation allows the system to preserve historical data while efficiently updating only necessary portions.
Solution Approach 2:
The patent performs preliminary actions by creating versioned copies of data chunks before updates. Each version is tagged with metadata indicating its creation time and contents. This preliminary versioning allows subsequent updates to reference previous versions rather than rewriting entire datasets, improving incremental update efficiency while maintaining data fidelity through version history.
2Reliability
If existing replication techniques are used, then data can be copied to object storage, but immutability support is lost
Solution Approach 1:
The patent applies preliminary action by creating versioned copies of data before any modifications. Each version is immutably stored with its creation timestamp and content hash. This preliminary versioning establishes an immutable history that prevents later modifications to previous versions, ensuring data integrity while the versioning mechanism itself manages the complexity of maintaining immutability.
Solution Approach 2:
The patent uses copying by creating new versioned copies of data chunks rather than modifying existing ones in place. Each copy is a complete or incremental snapshot that can be independently stored and referenced. This copying approach ensures immutability since original versions are never modified, only new versions are created, simplifying the enforcement of immutability guarantees.
3Reliability
If full data copying is performed for each update, then data fidelity is maintained, but storage overhead increases
Solution Approach 1:
The patent segments data into manageable chunks that can be independently versioned and updated. Instead of copying entire datasets on each update, only the affected chunks are copied to new versions. This segmentation dramatically reduces storage overhead while maintaining data fidelity through precise chunk-level versioning and referencing.
Solution Approach 2:
The patent applies selective copying by copying only the changed portions of data (chunks) rather than entire datasets. Unchanged portions are referenced from previous versions through versioned pointers. This selective copying approach maintains data fidelity for updated portions while avoiding redundant storage of unchanged data, significantly reducing overall storage overhead.
Data Source
AI summary
Write-once-read-many in cloud for file systems backed up to object storage (e.g., using a computerized tool), is enabled. For example, a system can comprise at least one processor, and at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations. The operations can comprise identifying changes to data in a file of a file system relative to a first version of an object chunk representative of the data in an object store, wherein the data is determined to comprise a defined data retention policy, copying changed data, of the data, from the file system to a second version of the object chunk, and copying unchanged data, of the data, from the first version of the object chunk to the second version of the object chunk.