Multimap Hash Table Layout for Cache-Efficient Database Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems experience performance issues due to high cache miss rates when processing large volumes of data, leading to inefficient query execution.
Innovation Solution
Implementing a hash table with ordered keys and a multimap data structure to optimize data storage and retrieval, using techniques that reduce cache misses and improve query performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional hash tables are used for storing database data in memory, then data access speed is improved, but cache miss rates increase leading to poor query performance
Solution Approach 1:
The patent applies preliminary action by pre-sorting the keys in the hash table before data access operations. This pre-organization of data in sorted key order enables sequential access patterns that improve cache utilization and reduce cache misses during query processing, thereby resolving the contradiction between fast data access and high query performance
2Ease of manufacture
If data is stored in unsorted hash table order, then insertion operations are simplified, but cache efficiency deteriorates due to random access patterns
Solution Approach 1:
The patent applies local quality by maintaining different organizational characteristics in different parts of the data structure. The hash table preserves local insertion simplicity while introducing global sorted key ordering. This allows the system to have both easy insertion operations and cache-efficient sequential access patterns by organizing keys in sorted order without complicating the insertion mechanism
3Productivity
If multiple processors are used to handle large data volumes, then processing capacity is increased, but system complexity and coordination overhead increase
Solution Approach 1:
The patent applies segmentation by dividing the large data set into multiple partitions that can be processed by different processors. Each processor handles a segment of the sorted keys, allowing parallel processing while maintaining overall sorted order. This segmentation increases processing capacity for large data volumes while keeping individual processor tasks manageable and reducing coordination overhead compared to unsegmented approaches
Data Source
AI summary
A database system uses a hash table for processing database queries. The system stores keys mapped to data values in the hash table. For processing a set of records using the hash table, the system partially sorts the records to match the order of keys of the hash table. The partial storing of the records results in improving the performance of a cache storing the hash table. The hash table may be a multimap that maps a key to multiple data values. The system stores data values mapped to a key in a vector that stores all data values mapped to the key in contiguous memory. The system stores a pointer to the vector in association with the key in the multimap. The system optimizes the multimap to store the data value instead of the pointer if there is a single data value associated with a key.