File System Sync Event Validation for Cloud Data Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud file storage systems face challenges in synchronizing local and remote file systems reliably, often leading to data loss and corruption due to race or error conditions during synchronization processes, which can result in improper deletes and confusion about the most recent file system state.
Innovation Solution
A system and method that validates file and folder deletes during synchronization by generating events for a steady-state synchronization process, comparing changes between local and remote file systems, and fencing invalid events to prevent them from being synchronized, ensuring that only valid changes are applied to both systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronization process is implemented to keep local and remote file systems updated, then data consistency between systems is improved, but data loss and corruption can occur due to race or error conditions
Solution Approach 1:
The patent applies preliminary action by generating and storing events for steady-state synchronization before performing rescan synchronization. These pre-generated events serve as a reference to validate delete operations during rescan, preventing premature or incorrect deletions. The system prepares validation data in advance (events from steady-state sync) that can be used to verify the correctness of delete operations during the rescan process, thereby preventing data loss before it occurs.
Solution Approach 2:
The patent implements feedback by validating rescan delete events against previously generated steady-state synchronization events. The system checks whether delete events detected during rescan are confirmed by corresponding events in the steady-state event log. This feedback mechanism allows the system to identify and fence (block) invalid delete operations that would otherwise cause data loss, ensuring that only verified delete operations are applied to the remote file system.
2Measurement precision
If rescan synchronization is performed to detect and correct synchronization errors, then synchronization accuracy is improved, but invalid delete events can cause unintended data loss
Solution Approach 1:
The system performs preliminary action by generating events during steady-state synchronization that record all file system operations. These pre-recorded events create an authoritative log of what operations should have occurred. During rescan synchronization, this pre-generated event log serves as the ground truth against which detected delete operations are validated, enabling the system to distinguish between valid and invalid deletes with high precision.
Solution Approach 2:
The patent uses an intermediary mechanism by introducing an event validation layer between the rescan delete detection and the actual delete execution. The steady-state events act as an intermediary reference that mediates the validation process. When a delete event is detected during rescan, the system consults the intermediary event log to verify its validity before applying the delete operation, thereby preventing unintended data loss while maintaining synchronization accuracy.
3Reliability
If events are validated against steady-state synchronization to prevent invalid deletes, then data integrity is improved, but synchronization process complexity increases
Solution Approach 1:
The patent applies copying by creating a copy of the file system state through event logging during steady-state synchronization. Instead of directly comparing file system states during rescan (which would be complex), the system copies the operational history into an event log. This event copy serves as a simplified reference structure that can be efficiently queried to validate delete operations, reducing the complexity of the validation process while maintaining data integrity.
Solution Approach 2:
The system discards invalid delete events by fencing them (blocking their execution) when validation fails. The patent recovers from potential data loss by identifying invalid deletes through validation and preventing their execution. This discard and recover mechanism allows the system to maintain simplicity in the validation logic (just check event presence) while ensuring data integrity through the fencing of invalid operations.
Data Source
AI summary
The invention prevents improper deletes of file system objects during rescan synchronizations of local and remote (cloud) file systems. File and folder deletes are validated (confirmed) during a rescan synchronization using events generated for a steady state synchronization process. Any improper deletes are fenced and not allowed to proceed during synchronization. A method for synchronizing local (LFS) and remote (RFS) file systems includes monitoring the LFS for changes, generating a first set of events as part of a first synchronization routine (e.g., a steady state synchronization routine), generating a second set of events as part of a second (rescan) synchronization routine, validating selected events of the second set with events of the first set to determine whether each of the selected events of the second set is valid or invalid, and synchronizing the file systems based on the events of the second set excluding those that have been invalidated. Storage systems are also disclosed.


