Per-File Synchronization Policies for Distributed Data Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional distributed data management systems are inadequate for environments with fewer nodes, unstable networks, and limited human administration, such as vehicle-based systems and IoT devices, as they struggle with data synchronization and reconciliation in scenarios involving unplanned power changes and multiple nodes joining simultaneously.
Innovation Solution
A distributed data management system that selects data synchronization and reconciliation policies on a per-directory or per-file basis, using versioning management data to determine whether to write updates, allowing for high availability or quorum-based schemes depending on the specific requirements of each file or directory, and employing a tree-based version management scheme for automated data reconciliation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a one size fits all synchronization policy is used, then the system is simple to manage, but it cannot adapt to different operating conditions and environments
Solution Approach 1:
The patent applies local quality by enabling different synchronization policies to be applied to different files or directories based on their specific requirements. Each file or directory can have its own synchronization settings (e.g., quorum-based, availability-based, or no synchronization) rather than applying a uniform policy across the entire system. This resolves the contradiction by allowing adaptability to diverse operating conditions while maintaining manageable complexity through granular, file-level control.
Solution Approach 2:
The system dynamically adjusts synchronization behavior based on real-time conditions such as network connectivity status, node availability, and file-specific requirements. The synchronization policy can change from static to dynamic by monitoring system state and adapting accordingly, enabling the system to respond to changing environments without requiring complex manual reconfiguration.
2Reliability
If quorum-based synchronization is used, then data integrity is maintained, but data loss occurs during reconciliation when nodes are disconnected
Solution Approach 1:
Different synchronization policies can be applied to different files based on their importance and requirements. Critical files can use quorum-based synchronization to maintain data integrity, while less critical files can use availability-based synchronization that allows local updates even when disconnected, preventing data loss during reconciliation. This selective approach resolves the contradiction by matching the synchronization strategy to the specific needs of each file.
Solution Approach 2:
The system can dynamically switch between different synchronization modes based on network conditions and file requirements. When nodes are disconnected, the system can adopt a more permissive synchronization policy that allows local updates to continue, preventing data loss. When connectivity is restored, the system can transition back to stricter quorum-based synchronization to ensure data integrity, thus resolving the contradiction between maintaining integrity and preventing data loss.
3Reliability
If centralized data storage is used, then data management is simple, but the system is susceptible to single point of failure
Solution Approach 1:
The patent segments data management by applying different synchronization and reconciliation policies to different files or directories. This segmentation allows the system to distribute management responsibilities and reduce dependency on a single centralized controller, thereby improving tolerance to node failure while keeping individual file management simple through automated, policy-driven operations.
Solution Approach 2:
The system implements self-service through automated synchronization and reconciliation processes that occur without human intervention. Nodes automatically synchronize data based on predefined policies, perform data reconciliation when disconnected, and manage their own state without requiring centralized administration. This resolves the contradiction by achieving high reliability through automated distributed management while avoiding the complexity of manual centralized control.
4Extent of automation
If automated data reconciliation is implemented, then human administration is reduced, but data loss may occur during synchronization
Solution Approach 1:
Different automated reconciliation policies can be applied to different files based on their specific requirements and risk profiles. Files with critical data can have stricter reconciliation policies that prevent data loss, while less critical files can use more aggressive synchronization strategies. This localized approach enables extensive automation while minimizing data loss risks through policy-specific control.
Solution Approach 2:
The automated reconciliation process dynamically adjusts its behavior based on system state, network conditions, and file requirements. The system can pause or modify reconciliation operations when conditions indicate potential data loss risk, and resume or intensify automation when conditions are favorable. This dynamic control enables high automation while preventing data loss through real-time adaptive decision-making.
Data Source
AI summary
A computing device includes an interface configured to communicate with a second computing device, a memory configured to store a first copy of a file that is synchronized with a second copy of the file stored at the second computing device, and a processor. The processor is configured, in response to receiving an update to the file while the interface is unable to communicate with the second computing device, to access versioning management data indicating synchronization settings associated with each of a plurality of files that includes the file. The processor is configured to, conditioned on determining that the versioning management data indicates a first synchronization setting associated with the file, write the update to the first copy, and conditioned on determining that the versioning management data indicates a second synchronization setting associated with the file, refrain from writing the update to the first copy.