Multi-master replication using chunk metadata for consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems face challenges in replication and processing updates to database tables, leading to difficulties in maintaining accurate and efficient data retrieval across multiple instances.
Innovation Solution
A multi-master replication system is implemented, where each database instance acts as both a client and server, exchanging modified records in the form of chunks with metadata, allowing for reliable and efficient data replication across peer instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional database replication methods are used, then data consistency across instances is maintained, but replication overhead and processing complexity increase
Solution Approach 1:
The patent segments replication updates into discrete chunks with metadata, where each chunk represents a unit of change detection and transport. This segmentation allows instances to process only relevant changes rather than entire datasets, reducing replication overhead while maintaining consistency.
Solution Approach 2:
The patent introduces chunk metadata as an intermediary mechanism that mediates between data changes and replication processes. The metadata serves as a compact representation that enables efficient change detection and coordination without requiring direct processing of all underlying data, thus reducing complexity.
2Adaptability or versatility
If multi-master replication is implemented, then data availability and access flexibility improve, but conflict detection and resolution complexity increases
Solution Approach 1:
The patent implements feedback mechanisms where instances exchange chunk metadata to detect conflicts and coordinate updates. The metadata includes information about changes made, allowing instances to detect conflicts and resolve them through coordinated feedback loops, managing multi-master complexity systematically.
Solution Approach 2:
The patent performs preliminary actions by exchanging chunk metadata before actual data updates occur. This preliminary exchange allows instances to detect potential conflicts in advance and coordinate update timing, preventing conflicts rather than resolving them after occurrence, thus reducing resolution complexity.
3Measurement precision
If frequent data synchronization is performed, then data freshness across instances is improved, but network bandwidth and processing time are consumed
Solution Approach 1:
The patent extracts only the essential change information into compact chunk metadata, separating this from the full data payload. Instances exchange only the metadata to detect changes, and only transfer actual data when necessary, significantly reducing network bandwidth consumption while maintaining data freshness through frequent lightweight synchronization.
Solution Approach 2:
The patent performs partial synchronization actions by exchanging only the metadata portion of chunks rather than complete data sets. This partial action enables frequent synchronization checks with minimal bandwidth consumption, allowing instances to maintain freshness awareness without the full cost of complete data transfers.
Data Source
AI summary
A multi-master replication system is disclosed. The multi-master replication system allows a large set of peer instances to collaboratively replicate data to each other. According to an example, a change detection thread running on a first server associated with a first instance of multiple instances of a replicated database monitors for changes to any of multiple records within one or more shared tables of the replicated database. Responsive to detection of a change to a record, an item is stored by the change detection thread onto a queue containing information regarding the change. Groups of changes are packaged into multiple chunks, in which each chunk (i) corresponds to a discrete unit of progress for both change detection and transport; (ii) is associated with multiple changed records; (iii) contains metadata about the multiple changed records; and (iv) does not contain data from the one or more shared tables.