Dedicated and Shared Log Buffers for Parallel Database Thread Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In database management systems with multiple processor cores, the high frequency of parallel log outputs leads to processing conflicts and decreased parallelism due to exclusivity delays and insufficient loading of logs in log buffers, resulting in a large number of writes to log files.
Innovation Solution
The system employs dedicated and shared log buffers, where dedicated buffers are used for threads updating non-shared DB regions and shared buffers for threads updating shared DB regions, allowing for efficient aggregation of logs and reducing conflicts during log file writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple threads perform parallel log outputs to avoid processing conflicts, then processing parallelism is improved, but the number of writes to log files increases due to insufficient log buffering
Solution Approach 1:
The system performs preliminary actions by accumulating logs in dedicated log buffers for each thread before writing to log files. This advance accumulation allows multiple logs to be prepared in parallel without immediately writing to files, reducing the total number of write operations while maintaining processing parallelism.
Solution Approach 2:
Dedicated log buffers serve as intermediary structures between threads and log files. These buffers mediate the write operations by collecting logs from multiple threads before transferring them to log files, thereby reducing the frequency of direct writes to log files while preserving parallel processing capabilities.
2Quantity of substance
If logs are accumulated in log buffers before writing to log files, then the number of writes is reduced, but exclusivity delays increase due to buffer contention
Solution Approach 1:
The log buffering system is segmented into multiple dedicated log buffers, one for each thread. This segmentation eliminates contention between threads for buffer access, as each thread exclusively uses its own dedicated buffer. Threads can accumulate logs in their respective buffers without waiting for others, reducing exclusivity delays while still reducing the total number of writes to log files through batch operations.
3Ease of operation
If dedicated log buffers are allocated to each thread, then buffer contention is reduced, but memory usage increases due to duplicate buffer structures
Solution Approach 1:
Each dedicated log buffer is designed to be multi-functional, serving both as a thread-specific accumulation buffer and as a temporary storage structure that can be flushed to log files. This universal design allows the same buffer structure to fulfill multiple roles, reducing the need for additional specialized memory structures and optimizing memory utilization while maintaining thread isolation.
Data Source
AI summary
A computer system according to the present invention has a plurality of processor cores that execute a plurality of threads in a parallel manner. When one of the threads updates a database (DB) region that is among a plurality of database regions associated with a database (DB), and that cannot be updated by more than one thread, the computer system selects one of one or more dedicated log buffers as a log buffer to which a log relating to the update of the database region is to be written. In contrast, when one of the threads updates a database region that is among the plurality of database regions, and that can be updated by more than one thread, the computer system selects one of one or more shared log buffers as a log buffer to which a log relating to the update of the database region is to be written. Each dedicated log buffer is a log buffer that can store one or more logs associated with only one thread and that cannot store two or more logs associate with two or more threads. Each shared log buffer is a log buffer that can store two or more logs associated with two or more threads.