Hashed Memory-Bank Storage for Rapid Database Joins
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems perform join functions and supply aggregation statistics in a suboptimal fashion.
Innovation Solution
Data from two or more columns of the build side of the database table are stored across multiple memory banks, allowing parallel access and processing through engines like Field Programmable Gate Arrays or Application Specific Integrated Circuits, with key columns hashed to assign rows to memory banks and compute aggregation statistics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored in traditional sequential memory structures, then data storage is simple, but database join functions and aggregation statistics computation are slow
Solution Approach 1:
The patent divides the build side table data into multiple independent memory banks, with each bank storing a portion of the hashed key data. This segmentation enables parallel access by multiple engines simultaneously, dramatically improving database join function speed while managing complexity through structured organization of the segmented data across banks.
2Productivity
If data is accessed sequentially from a single memory location, then memory access is simple, but parallel processing capability is limited
Solution Approach 1:
The patent introduces a new dimension of parallelism by organizing data across multiple independent memory banks that can be accessed simultaneously by different engines. The hash function maps build side keys to specific memory banks, creating a distributed memory architecture that enables true parallel processing while maintaining manageable access patterns through the hash-based routing mechanism.
3Productivity
If all build side data is loaded into memory for rapid access, then join function speed improves, but memory usage increases
Solution Approach 1:
The patent segments the build side table data across multiple memory banks, allowing selective loading of only the necessary portions into memory based on the specific join operation being performed. This segmentation enables the system to maintain rapid access speeds for the active data while avoiding the need to load entire large tables into memory simultaneously, thus optimizing memory resource utilization.
Data Source
AI summary
A system and method processes join requests via independently running engines. A build side table to be joined is allocated among high speed memories of the engines. Each row of a probe side are allocated to the engine likely to have build side data corresponding to the row, and the engine then performs the join. Aggregation statistics may be computed by distributing the information across the engines.