Database Row Existence Check Using Page Bitmaps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In-memory databases face inefficiencies in reconstructing state after shutdown due to high hash collisions when many rows are deleted, leading to slow lookup times and increased computation, as existing row-based designs result in extensive memory accesses during collision resolution.
Innovation Solution
Implementing a page-based data structure with a bitmap for each page, indexed by a hash of the page identifier, and using a sorted doubly-linked list to manage collisions, reducing memory accesses and improving both insertion and lookup times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If row-based data structure with hash indexing is used, then insertion speed is maintained, but lookup speed deteriorates when many rows are deleted due to hash collisions
Solution Approach 1:
The patent segments the flat row-based hash table into a hierarchical structure with page-level hash tables and row-level bitmaps. Each page contains multiple rows and has its own hash table entry, reducing the scope of hash collision resolution from entire row sets to smaller page-level structures, thereby improving lookup speed when many rows are deleted.
Solution Approach 2:
The patent adds a page dimension to the traditional row-based structure, creating a two-level hierarchy (page → row). This dimensional change allows the system to first locate the relevant page using page-level hashing, then quickly determine row existence within that page using a bitmap, avoiding the need to search through all rows with the same hash value.
2Productivity
If overflow data structure with linked list is used to resolve hash collisions, then insertion remains fast, but lookup becomes slow requiring comparison with every deleted row
Solution Approach 1:
The patent replaces the mechanical linked list traversal mechanism with a bitmap-based existence check. Instead of sequentially comparing row identifiers in a linked list, the system uses bitwise operations on a compact bitmap to instantly determine row existence, dramatically reducing lookup time while preserving insertion efficiency.
Solution Approach 2:
The patent changes the data representation parameter from storing actual row identifiers in overflow structures to storing compact bitmap flags. This parameter change transforms the lookup operation from linear comparison to constant-time bitwise checking, eliminating the time loss associated with linked list traversal.
3Ease of manufacture
If traditional row-based approach is used, then simple insertion is achieved, but extensive memory accesses are required during collision resolution
Solution Approach 1:
The patent merges the row existence tracking functionality into the page data structure itself using a bitmap, eliminating the need for separate overflow structures. This merging reduces memory access operations by keeping all necessary existence information localized within the page structure, accessible through single or few memory accesses rather than multiple accesses across scattered linked list nodes.
Data Source
AI summary
Example methods and systems are directed to existence checks on rows within a database. A page data structure stores bitmap data for the rows in the page. A row within the page corresponds to a single bit in the bitmap data. To determine if a row has been deleted, the page data structure for the page containing the row is located and the bit for the row in the page data structure is checked. An array of page data structures is created, indexed by a hash of the page identifier. In the event of a hash collision, a linked list of page data structures is used, sorted by page identifier.