Dynamic Log Management for Multi-Partition Transaction Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face complexity and inefficiency in data recovery due to the use of a fixed single log policy, particularly for multi-partition and non-deterministic transactions, which can lead to data errors and increased recovery time.
Innovation Solution
A dynamic log management method where a server node identifies transaction types and generates appropriate logs (command or data logs) based on transaction characteristics, allowing for flexible storage policies and asynchronous/synchronous storage approaches to optimize recovery speed and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a fixed single log policy is used for all transactions, then the system structure is simple, but data recovery complexity and time increase for multi-partition transactions
Solution Approach 1:
The patent implements dynamic log policy selection based on transaction type. The system determines whether to generate command logs, data logs, or both, depending on whether the transaction is single-partition or multi-partition. This dynamic approach allows the system to optimize log management for different transaction scenarios, reducing recovery time for multi-partition transactions while maintaining simplicity for single-partition transactions.
Solution Approach 2:
The patent applies different log management strategies to different transaction types. Multi-partition transactions receive comprehensive logging (both command and data logs) for accurate recovery, while single-partition transactions use simplified logging (command log only). This local differentiation optimizes the balance between recovery accuracy and system complexity for each specific case.
2Ease of manufacture
If a fixed single log policy is used for all transactions, then the system is easy to implement, but recovery accuracy decreases for multi-partition and non-deterministic transactions
Solution Approach 1:
The system dynamically adjusts the logging strategy based on transaction characteristics. By detecting whether a transaction is multi-partition or non-deterministic, the system automatically selects the appropriate log type (command log, data log, or both) to ensure accurate recovery while maintaining ease of implementation through automated detection and decision-making.
Solution Approach 2:
Different logging requirements are applied locally to different transaction types. Multi-partition and non-deterministic transactions receive enhanced logging for accurate recovery, while simple single-partition deterministic transactions use basic logging. This localized approach maintains overall system simplicity while ensuring high reliability where needed.
3Reliability
If data logs are generated for all transactions, then data recovery accuracy is maximized, but storage requirements and system complexity increase
Solution Approach 1:
The patent applies data logging selectively only to multi-partition and non-deterministic transactions that require it for accurate recovery. Single-partition deterministic transactions use command logging only. This selective approach maximizes recovery accuracy for complex transactions while minimizing storage requirements and system complexity for simple transactions.
Solution Approach 2:
Instead of generating data logs for all transactions (excessive action), the system generates data logs only when necessary (partial action) - specifically for multi-partition and non-deterministic transactions. This partial application of the more complex logging mechanism achieves sufficient recovery accuracy without the full burden of comprehensive logging.
4Quantity of substance
If command logs are used for all transactions, then storage requirements are minimized, but recovery time increases for multi-partition transactions
Solution Approach 1:
The system dynamically selects between command logging and data logging based on transaction type. For multi-partition transactions, data logging is activated to enable faster recovery by replaying only the necessary data changes. For single-partition transactions, command logging is used to minimize storage. This dynamic selection optimizes the balance between storage volume and recovery speed for each transaction type.
Solution Approach 2:
Different logging strategies are applied locally: data logging for multi-partition transactions (prioritizing recovery speed) and command logging for single-partition transactions (prioritizing storage efficiency). This localized differentiation allows the system to achieve fast recovery where needed while maintaining storage efficiency elsewhere.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides a log management method, a server, and a database system. The method includes: receiving, by a server, a first write transaction; generating and storing a command log of the first write transaction; detecting whether the first write transaction is a multi-partition transaction; and when detecting, by the server, that the first write transaction is a multi-partition transaction, generating and storing, by the server, a data log of the first write transaction. According to technical solutions disclosed in this application, a server node may use different log policies by determining a type of a write transaction. If the write transaction is a multi-partition transaction, a data log of the write transaction is generated and stored, and subsequently data recovery for the multi-partition transaction is performed by using the data log. In this way, data recovery efficiency is ensured, and a command log may be used to mark the multi-partition transaction, thereby avoiding extra marking information and reducing system complexity.