Distributed Version Reclaim Leader for Clustered Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In a database cluster, reference counting becomes complicated when determining when a version can be reclaimed, necessitating a need for efficient version reclaim (VR) that avoids duplication and communication failures.
Innovation Solution
Systems and methods extend reference count management to determine when a version is free across all database nodes, replicating cleanup transactions using transaction replication, and ensuring versions still in use are not reclaimed, while minimizing inter-node communication and avoiding invalid cleanup transactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If reference counting is used to determine when a version can be reclaimed in a database cluster, then version reclaim can be enabled, but the system complexity increases and communication overhead increases
Solution Approach 1:
The patent introduces a version reclaim leader as an intermediary component that coordinates version reclaim operations across the database cluster. This leader receives version release notifications from all database nodes, determines when versions can be safely reclaimed, and generates cleanup transactions. This intermediary approach simplifies the overall system by centralizing the complex decision-making logic for version reclaim, avoiding the need for each node to independently track and coordinate with all other nodes.
Solution Approach 2:
The patent merges the version reclaim management functionality into the existing transaction replication infrastructure. Cleanup transactions generated by the version reclaim leader are replicated to all database nodes using the same mechanism as regular transactions. This merging allows the system to leverage existing replicated log and transaction processing components, reducing overall system complexity while enabling distributed version reclaim.
2Loss of substance
If version reclaim is implemented in a database cluster, then storage space can be freed, but inter-node communication overhead increases
Solution Approach 1:
The patent extracts the version reclaim decision-making logic from the distributed nodes and consolidates it in the version reclaim leader. Database nodes only send lightweight version release notifications to the leader, rather than engaging in complex multi-node communication protocols. This extraction reduces communication overhead by minimizing the amount of data exchanged and the number of communication rounds required.
Solution Approach 2:
The system performs preliminary version release notifications asynchronously as versions become obsolete, rather than waiting for coordinated reclaim operations. The leader accumulates these notifications and batches the actual reclaim operations, performing cleanup only when conditions are favorable. This preliminary action allows the system to prepare for reclamation without immediate communication overhead.
3Productivity
If distributed version reclaim is implemented, then version cleanup can be performed across all nodes, but system reliability decreases due to communication failures
Solution Approach 1:
The patent implements beforehand cushioning by having database nodes buffer version release notifications locally before sending them to the version reclaim leader. If communication fails, nodes can resume sending buffered notifications when connectivity is restored. The leader similarly buffers received notifications and delays cleanup transactions until all necessary information is received, preventing premature reclamation due to temporary communication failures.
Solution Approach 2:
The system implements feedback mechanisms where the version reclaim leader tracks which nodes have sent release notifications for each version and only generates cleanup transactions when all nodes have confirmed release. This feedback loop ensures that reclamation decisions are based on complete information, preventing reliability issues from partial or lost communication.
4Measurement precision
If reference counting is extended to track versions across all database nodes, then accurate version reclaim determination can be made, but the device complexity and memory usage increase
Solution Approach 1:
The version reclaim leader acts as an intermediary that maintains the centralized tracking state for all versions across the cluster, rather than requiring each node to maintain its own complete tracking state. This intermediary approach achieves accurate measurement of version usage across the distributed system while concentrating the memory requirements in a single location, reducing total system memory usage.
Data Source
AI summary
Systems and methods for version reclaiming in a clustered system which avoid attempting to reclaim versions that are still in use by any member of the cluster. Version reclaim is performed efficiently by avoiding duplication of version reclaim effort, avoiding generating version reclaim cleanup transactions that are invalid (i.e., reclaiming versions that are still in use), and minimizing inter-database node communication.