Main Memory Database Log Compaction for I/O Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Main memory databases face performance degradation due to increasing log sizes when data is frequently changed, as existing solutions generate and store extensive change histories, leading to significant I/O operations and storage challenges.
Innovation Solution
A method for managing main memory databases that involves generating a first log of data changes and performing log compaction to create a second log containing only effective change information, which is then stored in a permanent device, thereby reducing log size and minimizing performance impact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log generation is performed for every data change to prevent data loss, then data reliability is improved, but log size increases indefinitely and I/O operations increase
Solution Approach 1:
The patent extracts only the essential change information from the complete data change history. Instead of logging every intermediate state, it identifies and retains only the effective changes that are necessary for data recovery, thereby reducing log size while maintaining data reliability.
Solution Approach 2:
The patent discards redundant log information that is not necessary for data recovery. By identifying and removing duplicate or superseded change records, it reduces the overall log size while preserving the essential information needed to recover data in case of loss.
2Reliability
If complete data change history is logged to restore data, then data recovery capability is improved, but I/O operations increase and performance decreases
Solution Approach 1:
The patent extracts only the necessary change information from the complete data change history. By identifying and retaining only the effective changes required for data recovery, it reduces the amount of I/O operations needed while maintaining full data recovery capability.
Solution Approach 2:
The patent applies partial action by logging only the essential portion of data changes rather than the complete history. This selective logging approach reduces I/O overhead and improves performance while still providing sufficient information for data recovery when needed.
3Reliability
If data backup is combined with log generation to store data, then data safety is improved, but storage space is consumed and I/O operations increase
Solution Approach 1:
The patent discards redundant information from the complete data change history, keeping only the essential changes needed for recovery. This reduces the storage space required for logs while maintaining data safety through effective change tracking.
Solution Approach 2:
The patent creates a streamlined copy of the change history that contains only the essential information needed for data recovery. This compressed log copy reduces storage requirements while preserving the critical data safety functionality.
Data Source
AI summary
An exemplary embodiment of the present disclosure discloses a management method of a main memory database which is performed in a computing apparatus including one or more processors and a main memory which stores commands executable in the processors. The management method of the main memory database includes: generating a first log including information of change, which includes at least one of a data insert, an update, and a deletion of data recorded in the main memory, in the main memory; recording the first log in a permanent storing device to generate a first log file; and performing log compaction on the first log to generate a second log including only effective change information among change information on data in order to reduce a size of the log.