Unified Framework for Global Table Replica Data Freshness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing databases lack flexibility and efficiency in providing multiple levels of database support, particularly in accommodating different levels of data freshness and disaster recovery requirements, often resulting in inadequate service and potential data disruption during operations like moving data between Paxos/RAFT groups.
Innovation Solution
A unified framework for a distributed database that allows for different levels of database support by using policies based on flushed log sequence numbers (LSN), applied LSN, and transaction timestamps to manage transactions across replica nodes, enabling efficient switching between support levels without bulk data movement and ensuring data freshness and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is moved between Paxos/RAFT groups to change database support levels, then database support flexibility is improved, but data availability and service continuity deteriorate due to substantial copying operations and potential data non-availability
Solution Approach 1:
The system segments database support levels into distinct policies (first policy, second policy, etc.) that can be independently applied to different tables. Each policy defines specific requirements for transaction commitment based on replica node status, allowing flexible configuration without moving data between groups.
Solution Approach 2:
The patent creates a universal framework where a single Paxos/RAFT group can serve multiple database support levels through policy configuration. The primary node can manage multiple tables with different policies within the same group, eliminating the need for separate groups for different support levels.
2Adaptability or versatility
If multiple Paxos/RAFT groups are used to provide different database support levels, then service quality is improved, but system complexity and operational overhead worsen due to managing multiple groups and data copying
Solution Approach 1:
The patent merges multiple database support levels into a single Paxos/RAFT group by implementing policy-based management. Different tables within the same group can have different policies applied, combining the functionality of multiple groups while reducing system complexity and eliminating data copying operations.
Solution Approach 2:
The system implements dynamic policy assignment where tables can be assigned to different database support policies without structural changes to the Paxos/RAFT group. Policies can be modified, added, or removed dynamically through metadata updates, allowing flexible adaptation without system reconfiguration.
3Reliability
If strict data freshness guarantees are implemented across all replica nodes, then data consistency is improved, but transaction commit speed and system performance worsen due to waiting for all replicas to catch up
Solution Approach 1:
The patent applies different freshness guarantee requirements to different tables based on their specific needs. Critical tables can have strict consistency requirements (waiting for all replicas), while non-critical tables can tolerate looser consistency (faster commit). Each table's policy is customized to its local quality requirements.
Solution Approach 2:
The system changes the consistency parameter dynamically based on policy configuration. Policies define different thresholds for replica synchronization (e.g., flushed LSN, applied LSN, transaction timestamps), allowing the system to adjust the strictness of consistency guarantees to optimize between data consistency and transaction commit speed.
Data Source
AI summary
The present disclosure provides for methods and systems related to a unified framework for global table with guarantee on replica data freshness. According to a first aspect, a method is provided. The method includes receiving a first transaction for updating a first table and a second transaction for updating a second table. The first and the second table being respectively associated with a first and second policy. The method further includes generating a first queue indicating the first transaction and a second queue indicating the second transaction. The method further includes receiving from a set of replica nodes information indicating a status of each replica with respect to the first transaction and the second transaction. The method further includes determining that at least one of policies is satisfied based on the received information and committing one of the transactions based on the determining