Dictionary Row Identifier Ordering for In-Memory Database Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In in-memory databases, traditional pushback dictionaries become inefficient as they store entries in the order written, leading to burdensome search times as the number of entries grows, hindering performance in large-scale database systems.
Innovation Solution
Implementing a forced ordering of row identifier values in dictionaries allows for quick read and lookup operations by maintaining row identifier values in a consecutive sequence, reducing the need to persist dictionary entries to main storage and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional pushback dictionaries are used to store row identifier values in write order, then the implementation is simple and easy to maintain, but the search time becomes burdensome as the number of entries grows
Solution Approach 1:
The patent applies preliminary action by sorting row identifier values in advance before storing them in the dictionary. This pre-sorting ensures that the dictionary maintains ordered entries from the beginning, enabling efficient binary search operations without requiring complex search algorithms later. The row identifier values are sorted in ascending order during insertion, which is a preliminary action that facilitates fast searching.
Solution Approach 2:
The patent creates a simplified copy of the dictionary structure that stores only essential row identifier mappings in sorted order. Instead of maintaining complex B-tree or hash table structures, the system uses a linear sorted array that copies only the necessary key-value pairs, reducing structural complexity while maintaining search efficiency through the sorted arrangement.
2Quantity of substance
If all dictionary entries are persisted to main storage, then data integrity is ensured, but storage space is consumed that could be used for other data structures
Solution Approach 1:
The patent extracts only the critical row identifier mappings that cannot be regenerated and persists them to storage. By identifying and separating the essential data (row identifier to value identifier mappings) from the transient data (consecutive sequence information), the system minimizes storage requirements while maintaining data integrity for the essential mappings.
Solution Approach 2:
The patent changes the persistence parameter from persisting all dictionary entries to persisting only non-consecutive row identifier mappings. This parameter change allows the system to store minimal data on disk while maintaining reliability, as the consecutive entries can be regenerated through mathematical relationships without losing data integrity.
3Quantity of substance
If row identifier values are stored in consecutive sequence, then storage space is optimized by not persisting all entries, but the system must efficiently track and reconstruct the sequence
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the starting row identifier value and the count of consecutive entries. This preliminary information allows the system to reconstruct any consecutive sequence without storing each individual entry, significantly reducing storage requirements while maintaining the ability to efficiently generate any needed value through simple arithmetic operations.
Solution Approach 2:
The patent enables the dictionary system to serve itself by implementing automatic reconstruction of consecutive entries through mathematical relationships. The system uses the stored base row identifier and count to self-generate any missing consecutive entries on demand, eliminating the need to persist every individual mapping while maintaining data availability through self-service reconstruction.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The subject matter disclosed herein provides methods for the forced ordering of a dictionary based on row identifier values. A plurality of concurrent transactions can be handled in an in-memory database. At least one of the transactions can include at least one write operation to a dictionary. Each write operation can be assigned a row identifier (ID). At least one of the write operations can be written to the dictionary out of sequence. The sequence can be based on the row ID. Each row ID in the dictionary can be mapped to a corresponding value identifier in the dictionary. The dictionary can position the value identifiers so that the corresponding row IDs are in a sorted sequential order based on the row ID. Related apparatus, systems, techniques, and articles are also described.