Database Write-Log Persistence for Lower Transaction Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current database systems experience excessively long latency in transaction processing due to the need for multiple database nodes to perform persistence operations, leading to increased end-to-end latency and persistence load.
Innovation Solution
The database system allows clients to persist write operation logs without waiting for the database nodes, reduces persistence requests by batching them, and enables exclusive storage for clients and nodes to enhance throughput and recovery speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If database nodes perform persistence operations before returning write responses, then data reliability is improved, but transaction processing latency increases
Solution Approach 1:
The database node performs persistence operation in advance before the client sends the next request, so that when the client needs to send the next write request, the persistence is already completed. This preliminary action eliminates the waiting time and reduces latency while maintaining reliability
Solution Approach 2:
The system uses asynchronous persistence where the database node autonomously completes the persistence operation in the background without blocking the client request. The client sends the next request as soon as it is ready, and the persistence is handled separately by the database node, making the system more efficient
2Stability of the object's composition
If multiple persistence operations are performed sequentially, then data consistency is improved, but persistence load increases
Solution Approach 1:
Instead of performing persistence operations sequentially for each write request, the system batches multiple persistence operations into periodic batches. Multiple write operation logs are collected and persisted together in batches, reducing the total number of persistence calls while maintaining data consistency
Solution Approach 2:
Multiple persistence operations are merged into batched persistence requests. The client collects multiple write operation logs and sends them to the storage resource in batches, combining multiple sequential persistence operations into fewer parallel operations, thereby reducing persistence load
Data Source
AI summary
This disclosure discloses a database system and a data management method therefor, and belongs to the field of database technologies. The database system includes a client, a plurality of database nodes, and a storage resource, and the plurality of database nodes include a first database node. The client is configured to send a first write request of a target transaction to the first database node, where the first write request indicates to-be-written first data. The first database node is configured to send a first write response to the client based on the first write request, where the first write response carries a first write operation log of the first data. The client is further configured to persist the first write operation log based on the storage resource. In this disclosure, a latency of processing the transaction by the client is reduced.