File System Sync Event Validation for Cloud Data Integrity

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing cloud file storage systems face challenges in synchronizing local and remote file systems reliably, often leading to data loss and corruption due to race or error conditions during synchronization processes, which can result in improper deletes and confusion about the most recent file system state.

Innovation Solution

A system and method that validates file and folder deletes during synchronization by generating events for a steady-state synchronization process, comparing changes between local and remote file systems, and fencing invalid events to prevent them from being synchronized, ensuring that only valid changes are applied to both systems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If synchronization process is implemented to keep local and remote file systems updated, then data consistency between systems is improved, but data loss and corruption can occur due to race or error conditions

Engineering Contradiction:
Improvedata consistencyVSAvoiddata loss
Core Design Contradiction:
ReliabilityVSLoss of substance

Solution Approach 1:

The patent applies preliminary action by generating and storing events for steady-state synchronization before performing rescan synchronization. These pre-generated events serve as a reference to validate delete operations during rescan, preventing premature or incorrect deletions. The system prepares validation data in advance (events from steady-state sync) that can be used to verify the correctness of delete operations during the rescan process, thereby preventing data loss before it occurs.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback by validating rescan delete events against previously generated steady-state synchronization events. The system checks whether delete events detected during rescan are confirmed by corresponding events in the steady-state event log. This feedback mechanism allows the system to identify and fence (block) invalid delete operations that would otherwise cause data loss, ensuring that only verified delete operations are applied to the remote file system.

Inventive Principle:
Principle #23Feedback

2Measurement precision

If rescan synchronization is performed to detect and correct synchronization errors, then synchronization accuracy is improved, but invalid delete events can cause unintended data loss

Engineering Contradiction:
Improvesynchronization accuracyVSAvoidunintended data loss
Core Design Contradiction:
Measurement precisionVSLoss of substance

Solution Approach 1:

The system performs preliminary action by generating events during steady-state synchronization that record all file system operations. These pre-recorded events create an authoritative log of what operations should have occurred. During rescan synchronization, this pre-generated event log serves as the ground truth against which detected delete operations are validated, enabling the system to distinguish between valid and invalid deletes with high precision.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent uses an intermediary mechanism by introducing an event validation layer between the rescan delete detection and the actual delete execution. The steady-state events act as an intermediary reference that mediates the validation process. When a delete event is detected during rescan, the system consults the intermediary event log to verify its validity before applying the delete operation, thereby preventing unintended data loss while maintaining synchronization accuracy.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Reliability

If events are validated against steady-state synchronization to prevent invalid deletes, then data integrity is improved, but synchronization process complexity increases

Engineering Contradiction:
Improvedata integrityVSAvoidsynchronization process complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies copying by creating a copy of the file system state through event logging during steady-state synchronization. Instead of directly comparing file system states during rescan (which would be complex), the system copies the operational history into an event log. This event copy serves as a simplified reference structure that can be efficiently queried to validate delete operations, reducing the complexity of the validation process while maintaining data integrity.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The system discards invalid delete events by fencing them (blocking their execution) when validation fails. The patent recovers from potential data loss by identifying invalid deletes through validation and preventing their execution. This discard and recover mechanism allows the system to maintain simplicity in the validation logic (just check event presence) while ensuring data integrity through the fencing of invalid operations.

Inventive Principle:
Principle #34Discarding and recovering

Data Source

PatentUS10437789B2System and method for delete fencing during synchronization of remote and local file systems
Publication Date: 2019.10.08 EGNYTE
  • US10437789B2 patent drawing
  • US10437789B2 patent drawing
  • US10437789B2 patent drawing

AI summary

The invention prevents improper deletes of file system objects during rescan synchronizations of local and remote (cloud) file systems. File and folder deletes are validated (confirmed) during a rescan synchronization using events generated for a steady state synchronization process. Any improper deletes are fenced and not allowed to proceed during synchronization. A method for synchronizing local (LFS) and remote (RFS) file systems includes monitoring the LFS for changes, generating a first set of events as part of a first synchronization routine (e.g., a steady state synchronization routine), generating a second set of events as part of a second (rescan) synchronization routine, validating selected events of the second set with events of the first set to determine whether each of the selected events of the second set is valid or invalid, and synchronizing the file systems based on the events of the second set excluding those that have been invalidated. Storage systems are also disclosed.