Cross-ACL Multi-Master Replication Conflict Resolution

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing multi-master replication systems face challenges in detecting data change conflicts, especially in access-controlled environments, where differences between missing information due to access control policies and deleted information are not adequately addressed, leading to complexities in data integrity and conflict resolution.

Innovation Solution

The implementation of cross-ACL multi-master replication, which uses per-access controlled data object version vectors to manage data changes across replication sites with different access control lists, ensuring appropriate application of data updates even when information is missing due to access control policies, and resolving conflicts by maintaining version vectors on a per-AC data unit basis.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If asynchronous multi-master replication is used to improve system availability, then system availability is improved, but data change conflicts occur due to concurrent changes at different sites

Engineering Contradiction:
Improvesystem availabilityVSAvoiddata change conflicts
Core Design Contradiction:
ReliabilityVSObject-generated harmful factors

Solution Approach 1:

The system performs preliminary actions by maintaining version vectors at each replication site that track the version history of data objects. Before applying data changes, the system checks version vectors to detect conflicts, and uses preliminary deconfliction rules to resolve conflicts automatically when possible, preventing harmful conflict propagation before it occurs.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms by propagating version vectors along with data changes between replication sites. Each site receives feedback about the version state of data at other sites, uses this feedback to detect conflicts, and adjusts its behavior accordingly by applying deconfliction rules or queueing changes until conflicts are resolved.

Inventive Principle:
Principle #23Feedback

2Difficulty of detecting and measuring

If version vectors are used to detect data change conflicts, then conflict detection capability is improved, but difficulty in detecting and measuring increases due to access control list complexities

Engineering Contradiction:
Improveconflict detection capabilityVSAvoidsystem complexity
Core Design Contradiction:
Difficulty of detecting and measuringVSDevice complexity

Solution Approach 1:

The system segments the version tracking mechanism by maintaining separate version vectors for different data objects and their attributes. Each version vector tracks versions independently, allowing the system to detect conflicts at granular levels (object level, attribute level) rather than requiring a monolithic version tracking approach, thereby managing complexity through modular organization.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system introduces version vectors as intermediary data structures that mediate between data changes and conflict detection. These version vectors serve as intermediaries that carry version information between replication sites, enabling conflict detection without requiring direct complex comparisons of entire data states, thus simplifying the detection process while maintaining accuracy.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Object-affected harmful factors

If access control lists are applied to filter data propagation between sites, then data security is improved, but information completeness deteriorates causing inability to distinguish missing information from deleted information

Engineering Contradiction:
Improvedata securityVSAvoidinformation completeness
Core Design Contradiction:
Object-affected harmful factorsVSLoss of information

Solution Approach 1:

The system performs preliminary action by maintaining version vectors that track the complete history of data objects including attributes that may be filtered by access control lists. This preliminary version tracking preserves information about what data existed and its version history, enabling later distinction between data that was filtered due to access control versus data that was actually deleted, thereby compensating for the information loss caused by ACL filtering.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by propagating version vector information along with data changes between replication sites. This feedback mechanism provides receiving sites with version history information that complements the data received through ACL-filtered channels, enabling them to distinguish between missing information (filtered by ACL) and deleted information (actually removed), thus recovering information completeness despite access control filtering.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS10936573B2Cross-ACL multi-master replication
Publication Date: 2021.03.02 PALANTIR TECHNOLOGIES INC
  • US10936573B2 patent drawing
  • US10936573B2 patent drawing
  • US10936573B2 patent drawing

AI summary

Techniques for cross-ACL multi-master replication are provided. The techniques allow a replication site in a multi-master replication system implementing an asynchronous replication protocol and an access control policy to appropriately apply received data change updates to data maintained at the site even where a data change update is missing information because of the implemented access control policy.