Interleaved Dictionary Blocks in Page Chain
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems face inefficiencies in storing and loading dictionaries for multiple columns, particularly in small tables with a large number of columns, due to the need to load entire page or page chains into memory, resulting in significant storage and input/output overhead.
Innovation Solution
The solution involves interleaving dictionary blocks of multiple columns within a single page or page chain, allowing each block to be assigned to a specific dictionary and loaded efficiently by scanning the page chain once, with each block containing a column ID, starting value ID, and type, enabling parallel construction of in-memory dictionaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If dictionaries for multiple columns are stored in separate pages or page chains, then each dictionary can be independently managed, but the storage overhead and I/O requirements increase significantly when loading data into memory
Solution Approach 1:
The patent combines multiple column dictionaries into a single shared page or page chain structure. Instead of allocating separate pages for each column's dictionary, the system interleaves multiple dictionary blocks within the same page, allowing efficient space utilization and reducing the total number of pages that need to be loaded into memory.
Solution Approach 2:
The patent segments the shared page into multiple interleaved dictionary blocks, each belonging to a different column. Each block contains a header with metadata (column ID, starting value ID, type) followed by the dictionary entries. This segmentation allows the system to manage multiple dictionaries within a unified structure while maintaining column-specific organization.
2Productivity
If entire pages or page chains are loaded into memory for database operations, then all dictionary data is available for processing, but the I/O overhead and memory consumption increase
Solution Approach 1:
By merging multiple column dictionaries into a single shared page, the system reduces the number of I/O operations required to load dictionary data. Instead of loading multiple separate pages, the system loads one shared page that contains all necessary dictionary blocks, significantly reducing I/O overhead and improving database operation performance.
3Quantity of substance
If multiple dictionary blocks are stored in a single page, then storage efficiency improves, but the complexity of managing and loading these interleaved blocks increases
Solution Approach 1:
The patent segments the shared page into clearly defined dictionary blocks, each with a standardized structure containing a header (column ID, starting value ID, type) and dictionary entries. This segmentation provides a systematic organization that simplifies management compared to a monolithic structure.
Solution Approach 2:
The shared page acts as an intermediary structure that organizes multiple column dictionaries in an interleaved manner. The page header and block headers serve as mediators that provide metadata and organization, making it easier to manage and load the interleaved blocks compared to separate page structures.
4Ease of operation
If separate pages are used for each column dictionary, then dictionary loading can be straightforward, but the load time and memory usage increase for small tables with many columns
Solution Approach 1:
The patent merges multiple column dictionaries into a single shared page, allowing the system to load all dictionary data in a single I/O operation rather than loading multiple separate pages. This significantly reduces the load time for small tables with many columns, as the entire dictionary structure fits within one page that can be loaded efficiently into memory.
Data Source
AI summary
A first block in a page chain can be assigned to a first dictionary for a first column of a database. The page chain can include a plurality of dictionary blocks allocated to a plurality of dictionaries of a plurality of columns including the first column such that the plurality of dictionary blocks is interleaved in the page chain. A new unique value to be added to the first column can be appended to an end of the first dictionary. The plurality of dictionaries can be loaded from the page chain into memory of one or more computers performing operations of a database management system. The loading can include scanning the page chain a single time and assigning the plurality of dictionary blocks to the respective dictionaries of the plurality of dictionaries.