Bitmap Filter Mapping for Collision-Free Database Joins
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database query execution in star schema systems is hindered by the inefficiencies of hash joins and Bloom Filters, leading to increased computational effort and resource usage due to hash collisions and unnecessary data processing.
Innovation Solution
Implement a one-to-one mapping between identifiers and bit locations in a bitmap filter, eliminating the need for hash function evaluations and reducing computational effort by ensuring the bitmap filter length covers only relevant identifiers, allowing for direct processing and caching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hash joins and Bloom Filters are used for database query execution, then query processing capability is maintained, but computational effort and resource usage increase due to hash collisions and unnecessary data processing
Solution Approach 1:
The patent extracts the essential filtering function from complex hash join operations and Bloom Filters, implementing it through a simplified bitmap-based approach. The bitmap filter directly maps identifiers to bit positions, eliminating the need for hash function evaluations and collision resolution while maintaining effective filtering capability.
Solution Approach 2:
The patent replaces the mechanical hash joining process with a deterministic bitmap indexing system. Instead of using hash functions that require computational evaluation and collision handling, the system uses direct mathematical mapping from identifiers to bit positions, substituting complex computational mechanics with simpler arithmetic operations.
2Measurement precision
If hash functions are evaluated and hash collisions are resolved, then data retrieval accuracy is maintained, but time consumption increases
Solution Approach 1:
Instead of computing hash values and resolving collisions to achieve accurate data retrieval, the patent inverts the approach by using direct mathematical mapping from identifiers to bit positions. This inversion eliminates the time-consuming hash evaluation and collision resolution steps while maintaining accurate data retrieval through deterministic bitmap lookup.
3Productivity
If Bloom Filters are used for filtering, then query results are obtained, but resources incurred in operating upon Bloom Filters increase
Solution Approach 1:
The patent replaces complex Bloom Filter structures with simpler bitmap data structures. The bitmap filter uses straightforward bit arrays that are easier to implement, manage, and process, eliminating the need for sophisticated Bloom Filter operations while maintaining effective filtering functionality for query execution.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
There is provided a computer-implemented method of generating a bitmap filter. A filter parameter is received, and a first data source associated with the filter parameter is queried to identify at least one entry in the first data source with an identifier corresponding to the filter parameter. A first process is performed where zero or one single bit location of a plurality of bit locations in a bitmap filter is identified that corresponds to an identifier of an entry of the first data source corresponding to the filter parameter. Each identifier has a numerical value and the bit location is identified based on the numerical value of the corresponding identifier. The single bit location is assigned to the identifier, such that there is one-to-one mapping between each identifier corresponding to the filter parameter and an assigned bit location in the bitmap filter. A bit is set at the assigned bit location. The first process is repeated for another entry in the first data source with an identifier corresponding to the filter parameter.