Range Deletes on Distributed Database Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed database systems face challenges in efficiently managing range deletes across multiple storage nodes, leading to increased complexity and overhead due to the need for atomic transactions and tombstone records management.
Innovation Solution
Implementing a two-phase commit protocol to prepare and commit tombstone records on each storage node, ensuring that either all items in a range are deleted or none are, and using tombstone records to efficiently manage data retrieval by treating items as deleted, thereby reducing overhead over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a two-phase commit protocol is used to ensure atomic range delete operations across distributed storage nodes, then reliability is improved, but device complexity increases due to tombstone record management
Solution Approach 1:
The patent introduces tombstone records as intermediary objects that mediate between the delete operation and the actual data removal. These tombstone records serve as placeholders that mark ranges of deleted items without requiring immediate physical deletion, thereby simplifying the coordination protocol while maintaining atomicity across distributed nodes.
Solution Approach 2:
The system performs preliminary action by creating tombstone records before actual data deletion occurs. This preliminary marking allows the system to establish the delete intent and maintain consistency across nodes, with the actual cleanup happening asynchronously, thus reducing the complexity of coordinated deletion operations.
2Productivity
If tombstone records are maintained to track deleted items across distributed nodes, then data retrieval efficiency is improved, but loss of time increases due to the cleanup process
Solution Approach 1:
The system implements periodic action through background cleanup processes that asynchronously remove tombstone records after their retention period expires. This allows query operations to immediately utilize tombstone records for efficient data retrieval, while cleanup occurs periodically without blocking normal operations, thus minimizing time loss.
Solution Approach 2:
Tombstone records enable continuous useful action by maintaining deleted item metadata available for query operations. The records persist and serve query purposes until cleanup, ensuring that data retrieval efficiency is maintained continuously while the cleanup process operates independently in the background.
3Reliability
If range delete operations are implemented atomically across multiple storage nodes, then reliability is improved, but productivity decreases due to increased overhead
Solution Approach 1:
The patent applies segmentation by dividing the delete operation into two independent phases: tombstone record creation and actual data deletion. This segmentation allows the atomic commitment phase to complete quickly with minimal coordination overhead, while the resource-intensive deletion phase occurs asynchronously, thereby improving overall throughput while maintaining reliability.
Solution Approach 2:
The system performs preliminary action by establishing tombstone records and committing the transaction before actual deletion occurs. This preliminary commitment phase is lightweight and can be executed rapidly across distributed nodes, improving productivity, while the actual deletion happens subsequently without blocking the transaction commit.
Data Source
AI summary
A distributed database system maintains data for a logical table by storing, on a plurality of storage nodes, a collection of key-item pairs. An operation to delete a range of these items is processed by first storing, on each of the plurality of storage nodes, a tombstone record indicative of the items to be deleted. The tombstones are committed once each of the plurality of nodes has stored the tombstone. Items within the range are treated as deleted. A cleanup operation deletes items within the range and updates the tombstone.