Segmented Bitmap Index Structure for Sparse Row ID Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database index tables face inefficiencies due to rapid growth of row IDs, leading to increased memory usage and storage requirements, especially when updates and deletions disrupt contiguity, causing large spans of unused storage in bitmap representations.
Innovation Solution
Implementing an index structure composed of segments that actively represent relevant row IDs and passively represent inactive ones through gaps between segments, allowing for efficient storage and management of row IDs without unnecessary storage consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a conventional index table stores all row IDs using bitmap representation, then complete tracking of row IDs is achieved, but storage space increases substantially
Solution Approach 1:
The index table is divided into multiple segments, where each segment tracks a specific range of row IDs. This segmentation allows the system to only allocate storage for segments that contain active row IDs, rather than maintaining a continuous bitmap for the entire row ID space. The patent implements this by creating segment objects that each manage a portion of the overall row ID range, enabling sparse representation when row IDs are not contiguous.
2Productivity
If row IDs increase rapidly in the database table, then more rows can be stored, but the memory required to store the index table increases substantially
Solution Approach 1:
The index structure uses segmented bitmaps where each segment corresponds to a range of row IDs. When row IDs increase rapidly, new segments are created only for the ranges that contain active rows, rather than expanding a single large bitmap. This allows the index table to scale with database capacity while maintaining efficient memory usage through lazy allocation of segment structures.
Solution Approach 2:
The patent transitions from a one-dimensional continuous bitmap to a multi-dimensional segmented structure. Each segment is a separate data structure that can be independently managed, allowing the system to navigate through multiple levels of organization (segments within an index table). This dimensional change enables efficient handling of sparse row ID distributions across large databases.
3Adaptability or versatility
If updates and deletions disrupt contiguity of row IDs, then database operations become more flexible, but large spans of unused storage are created in bitmap representations
Solution Approach 1:
When updates and deletions create gaps in row ID contiguity, the segmented structure automatically isolates these gaps to specific segments. Each segment independently manages its range, so gaps in one segment do not affect others. The system can then optimize individual segments by merging adjacent segments or removing empty segments, preventing the accumulation of unused storage across the entire index table.
Solution Approach 2:
The patent implements mechanisms to discard empty or fully consolidated segments and recover their storage space. When a segment becomes empty due to deletions or when segments can be merged to eliminate gaps, the system removes these segments from the index structure, thereby recovering the storage space that would otherwise be wasted in a continuous bitmap representation.
Data Source
AI summary
Representative embodiments enable the management of index structures for data tables within a database. The techniques can include (1) receiving a request to perform an operation (e.g., read, write, etc.) on a row identifier (ID) associated with an index structure, (2) identifying a plurality of segments that corresponds to the index structure, where each segment stores a respective bitmap, (3) identifying, based on the row ID, a logical block ID of a logical block that encompasses the row ID, (4) utilizing the logical block ID to identify, among the plurality of segments, a subset of segments encompassed by the logical block, and (5) parsing each segment of the subset of segments to identify a respective bitmap that encompasses the row ID, and (6) in response to identifying the respective bitmap: performing an operation on a bitmap value that corresponds to the row ID.