Database Page Tracking for OLAP Snapshot Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hybrid systems for online transaction processing (OLTP) and online analytical processing (OLAP) face performance issues due to the time-consuming process of forking large processes, which is exacerbated by the need to replicate a large number of page table entries, leading to periodic forking that impacts online data processing performance.
Innovation Solution
A method and apparatus that track and synchronize only the changed pages between a main process executing OLTP and a child process executing OLAP, using an operating system kernel to manage page changes and synchronize them at predetermined sync time points, optimizing the copy-on-write mechanism to only the delta changes, thereby avoiding periodic forking and enhancing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic forking is used to maintain consistent snapshots for OLAP queries, then data consistency for OLAP is improved, but system performance and response time deteriorate due to the time-consuming replication of page table entries
Solution Approach 1:
The patent extracts only the changed pages from the parent process and synchronizes them to child processes, rather than copying the entire process memory space. This selective extraction of changed data reduces the synchronization overhead while maintaining data consistency for OLAP queries.
Solution Approach 2:
The patent segments the memory space into individual pages and tracks changes at the page level. By identifying and synchronizing only the modified pages rather than the entire address space, the system achieves efficient updates that maintain consistency without the performance penalty of full process forking.
2Reliability
If full process forking is performed to create consistent snapshots, then OLAP query consistency is improved, but the time required for snapshot creation increases
Solution Approach 1:
The patent implements preliminary tracking of page changes using copy-on-write mechanisms before snapshots are needed. By monitoring which pages have been modified and preparing this information in advance, the system can quickly synchronize only the necessary changed pages when a snapshot is required, rather than copying everything.
Solution Approach 2:
The patent uses selective copying of only the changed pages to child processes rather than copying the entire process memory. This targeted copying approach maintains snapshot consistency while dramatically reducing the time required compared to full process forking.
3Stability of the object's composition
If page table entries are replicated during forking, then memory consistency between parent and child processes is improved, but the complexity of the forking operation increases
Solution Approach 1:
The patent introduces an intermediary mechanism that tracks page changes and manages synchronization between parent and child processes. This intermediary layer handles the complexity of page table management, allowing the system to maintain memory consistency without directly implementing complex forking logic at every synchronization point.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
Embodiment of the present invention discloses a method and an apparatus of maintaining data for online analytical processing in a database system. The method includes: tracking a changed page in a main process; and synchronizing the changed page to a child process for online analytical processing. In the method and apparatus of maintaining data for online analytical processing, the changed pages are tracked and then the child process is synchronized with the changed pages. Therefore, periodic forking is avoided, fork overhead due to periodic forking in the prior art is removed, the synchronization is faster since only the changed pages are synchronized, and the performance of online data processing is enhanced.