Database Load Unit Format Switching for Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face challenges in efficiently loading database objects into memory due to memory limitations, particularly when switching between column-loadable and page-loadable formats, which requires costly rewrites and consumes significant resources.
Innovation Solution
Implementing a unified persistence format that allows database objects to be loaded in either column-loadable or page-loadable formats without requiring persistence store rewrites, enabling flexible load unit conversion based on workload and operating conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If database objects are loaded into memory in column-loadable format, then query performance is improved, but memory consumption increases significantly
Solution Approach 1:
The patent implements dynamic format switching between column-loadable and page-loadable formats based on workload conditions. The system monitors memory availability and query patterns, automatically selecting the optimal load format. When memory is abundant and analytical queries dominate, column-loadable format provides high performance. When memory is constrained or transactional workloads prevail, the system switches to page-loadable format to reduce memory consumption.
Solution Approach 2:
The patent changes the load format parameter dynamically based on system state. By adjusting the format parameter (column-loadable vs. page-loadable) according to memory conditions and workload characteristics, the system optimizes both performance and resource utilization. This parameter change allows the same persistence store to serve different operational requirements without rewriting data.
2Quantity of substance
If database objects are loaded into memory in page-loadable format, then memory consumption is reduced, but query performance deteriorates
Solution Approach 1:
The system dynamically switches between page-loadable and column-loadable formats based on real-time conditions. When memory consumption is high or transactional workloads are dominant, page-loadable format conserves memory resources. When analytical queries become prevalent and memory is available, the system transitions to column-loadable format to maximize query performance, thus dynamically balancing memory usage and performance.
Solution Approach 2:
The load format parameter is adjusted based on system state and workload type. Page-loadable format is selected when memory conservation is priority, while column-loadable format is chosen when performance is critical. This parameter change mechanism enables the system to adapt to varying operational requirements without data rewrites.
3Adaptability or versatility
If the persistence store format is changed to switch between column-loadable and page-loadable formats, then load unit flexibility is improved, but data rewrite operations consume significant time and resources
Solution Approach 1:
The patent creates a universal persistence format that can be interpreted in multiple ways (column-loadable or page-loadable) depending on the load unit parameter. The same persisted data structure serves both loading modes without requiring separate storage formats or data rewrites. This multi-functionality eliminates the need for time-consuming data transformation while maintaining full flexibility in load unit selection.
Solution Approach 2:
Instead of physically rewriting data when changing load formats, the system uses logical copying or interpretation of the same persistence store. The persistence layer remains unchanged, while the loading logic adapts to different formats by reading the same data in different ways. This avoids expensive data rewrite operations while providing format flexibility.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A primary database system loads database objects into a primary in-memory store according to a given format. The primary database captures, in replay logs, the loading of the database objects according to the given format. The primary database sends the replay logs to a secondary database system. In response to receiving a replay log, the secondary database checks the value of a log replay configuration parameter. If the configuration parameter is a first value, the secondary database replays the replay log to load the corresponding database objects into a secondary in-memory store according to a first format. If the configuration parameter is a second value, the secondary database replays the log to load the objects according to a second format, and if the configuration parameter is a third value, the secondary database replays the log to load the objects in a same format which was used by the primary database.