Database Copy Pages for Hot-Page Flush Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inconsistent data between data pages on read-write nodes and disks in databases due to delayed flush page processing, leading to thread blocking and performance issues, particularly with frequently modified 'hot' pages.
Innovation Solution
Implementing a copy page mechanism that includes sequence identifiers and data information for data pages that do not meet disk write conditions, allowing for timely flush page processing and log deletion in read-only nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data pages are frequently modified on read-write nodes, then data processing capacity and speed are improved, but inconsistency between data pages and disk increases, causing thread blocking
Solution Approach 1:
The system performs preliminary actions by generating copy pages in advance when data pages are first modified, capturing the state at that moment. This allows the original data pages to be flushed to disk without waiting for subsequent modifications to complete, while the copy pages serve as backups that can be applied later to read-only nodes, thus preventing thread blocking and maintaining data consistency.
Solution Approach 2:
The invention segments the data page management into two independent parts: original data pages on read-write nodes and copy pages. This segmentation allows the original data pages to be flushed to disk independently without being blocked by frequent modifications, while copy pages capture snapshots for replication to read-only nodes, resolving the contradiction between processing speed and data consistency.
2Reliability
If flush page processing is performed for every data page modification, then data consistency is maintained, but thread blocking occurs and performance deteriorates
Solution Approach 1:
Instead of flushing every modified data page to disk, the system creates copy pages that capture the state of data pages at specific moments. These copy pages are then replicated to read-only nodes. This copying mechanism maintains data consistency across nodes without requiring frequent flush operations on read-write nodes, thus avoiding thread blocking and performance deterioration.
Solution Approach 2:
The system performs preliminary copying of data page states to copy pages before disk flush operations are needed. This preliminary action captures the necessary data state information in advance, allowing subsequent disk writes to proceed without blocking threads that continue to modify data pages.
3Reliability
If copy pages are generated for all data pages, then data consistency is ensured, but system complexity and resource consumption increase
Solution Approach 1:
The system applies copy page generation selectively rather than universally. Copy pages are generated based on specific conditions such as when data pages are first modified or when they meet certain flushing criteria. This localized application of copy page creation reduces unnecessary system complexity and resource consumption while still ensuring data consistency where needed.
Data Source
AI summary
A data processing method, a database system, and an electronic device are disclosed. The method includes: generating a copy page when a data page does not satisfy a condition for writing into a disk; wherein the copy page includes: a first sequence identifier of a log corresponding to modification of the data page at a first time, a second sequence identifier of a log corresponding to modification of the data page at a second time, and data information of the data page after multiple modifications over a period from the first time to the second time, the first time being earlier than the second time; and executing an operation of updating the first sequence identifier corresponding to the data page in the disk to the second sequence identifier when the copy page satisfies the condition for writing into the disk.