Bloom Filter Index Using Hamming Distance for Multi-Property Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Relational databases face inefficiencies in searching and indexing large datasets due to high overhead in maintaining multiple indexes, which can lead to full table scans and intrusive index maintenance, especially in extremely large systems, where a single index capable of detecting objects based on multiple properties is advantageous.
Innovation Solution
Implementing a Bloom filter index system that uses Hamming values and binary logarithms of Bloom filters to efficiently search for objects with specified properties, allowing a single index to cover multiple data columns and reduce the number of index scans required for matching, while mitigating false positives through filtering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple indexes are created to search for various combinations of column values, then search capability is improved, but index maintenance overhead and storage space increase
Solution Approach 1:
The patent combines multiple column values into a single composite Bloom filter by hashing each column value and setting corresponding bits to 1. This merging approach allows a single index structure to represent multiple columns, reducing the number of indexes from many individual column indexes to one composite index, thereby decreasing maintenance overhead while preserving multi-column search capability
Solution Approach 2:
The composite Bloom filter index serves multiple functions simultaneously: it can search for objects based on any single column or any combination of columns. A single index structure provides universal search capability across all indexed columns, eliminating the need for separate specialized indexes for each column or column combination
2Device complexity
If a single Bloom filter index is used to search for objects with multiple properties, then index maintenance is simplified, but search efficiency may decrease due to false positives
Solution Approach 1:
The patent segments the Bloom filter verification process into two distinct phases: first, a rapid filtering phase using Hamming distance to identify candidate objects that could potentially match, and second, a precise verification phase using full Bloom filter comparison. This segmentation allows the system to quickly eliminate non-matching objects while maintaining accurate final results, balancing speed and reliability
Solution Approach 2:
The patent performs preliminary filtering using Hamming distance calculation before conducting the final Bloom filter match verification. By pre-screening objects based on their Hamming distance to the query Bloom filter, the system eliminates clearly non-matching objects early in the process, reducing the number of expensive full Bloom filter comparisons needed while maintaining search accuracy
3Ease of manufacture
If linear search is used to locate a target Bloom filter in a collection, then implementation is simple, but search time increases for large datasets
Solution Approach 1:
The patent transforms the one-dimensional linear search problem into a multi-dimensional search space by incorporating Hamming distance as an additional dimension for filtering. Instead of sequentially comparing Bloom filters one by one, the system uses Hamming distance to create a geometric filtering dimension that rapidly eliminates regions of the search space that cannot contain matches, reducing the effective search area while maintaining implementation feasibility
Data Source
AI summary
Identifying objects in a datastore with specified object properties, where each object is characterized by a Bloom filter, a Hamming value of the Bloom filter, and a binary logarithm of the Bloom filter. A set of object properties is received. A search Bloom filter is created from the object properties. A Hamming value of the search Bloom filter is determined. A binary logarithm of the search Bloom filter is calculated. Objects in the datastore which have respective Hamming values greater than or equal to the Hamming value of the search Bloom filter and respective binary logarithms that are greater than or equal to the binary logarithm of the search Bloom filter are identified.