Distributed Database PiT Creation With Logical Timestamps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for creating, maintaining, and removing point-in-time images (PiTs) in distributed databases are not efficient, scalable, and slow down other activities in a non-linearly scalable fashion, especially in clusters with a large number of nodes.
Innovation Solution
A system and method for efficiently creating PiTs in a distributed database system using a sequencer server (SEQ-server) that provides a logical timestamp counter (LTC) value as a timestamp for PiT creation, allowing for high-rate PiT creation and commitment processes without synchronization among nodes, utilizing a logical timestamp counter (LTC) and commitment timestamp (CMTS) to manage transaction states and content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional PiT creation techniques are used in distributed databases, then PiT creation can be achieved, but the system becomes slow and non-linearly scalable, especially in clusters with large number of nodes
Solution Approach 1:
The system segments the PiT creation process by introducing a dedicated SEQ-server that handles timestamp generation separately from the main database nodes. This segmentation allows the database nodes to focus on data operations while the SEQ-server manages the complex coordination of timestamps across distributed nodes, thereby improving scalability and PiT creation rate.
Solution Approach 2:
The SEQ-server acts as an intermediary component between database nodes and the PiT creation process. It receives requests from nodes, generates appropriate timestamps using the logical timestamp counter, and returns them to the nodes. This intermediary role eliminates the need for nodes to directly coordinate with each other for timestamp generation, reducing system complexity and enabling linear scalability.
2Reliability
If concurrency control protocols are introduced to ensure transaction consistency, then data correctness is improved, but performance reduction occurs due to additional complexity and overhead
Solution Approach 1:
The system uses logical timestamps (LTCs) as virtual copies of time information to track transaction states. Instead of using complex locking mechanisms or distributed consensus protocols, the system creates PiTs with LTC values that serve as copies of the transaction state at a given time. This allows concurrent transactions to proceed independently while maintaining consistency through the timestamp-based ordering, significantly reducing overhead and improving performance.
3Measurement precision
If synchronization among nodes is required for PiT creation, then point-in-time consistency is achieved, but processing time increases and high-rate PiT creation becomes difficult
Solution Approach 1:
The SEQ-server performs preliminary action by pre-generating and assigning logical timestamps to transactions before they actually execute. The LTC is incremented and assigned in advance based on the request arrival order at the SEQ-server, eliminating the need for synchronization during transaction execution. This preliminary timestamp assignment ensures point-in-time consistency while adding minimal time overhead, enabling high-rate PiT creation.
Data Source
AI summary
A method and system for managing execution of database transactions by facilitating point-in-time image (PIT). The method comprises receiving at least one statement being part of a transaction, the transaction is initiated by a client to be executed on a distributed database system; sending a single request to a sequencer server (SEQ-server) to create at least one PiT, wherein the SEQ-server provides a logical timestamp counter (LTC) value as a timestamp of the PiT; upon creation of the at least one PiT, causing execution of the at least one statement, a PiT provides a state and content of committed data in the distributed database system; upon receiving a commit statement, performing a commitment process of the transaction, the commitment process is based on a commitment timestamp (CMTS) set by the SEQ-server based on an incremented value of the LTC; and returning an acknowledgment of the transaction commitment to the client.