Hash-Based Set Operations in Database Management Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems face inefficiencies in performing set operations due to the need for sorting large data sets, which significantly contributes to latency and computational overhead, especially when sets are not pre-sorted.
Innovation Solution
Implementing a hash-based approach that eliminates the need for sorting by using hash maps to efficiently combine and aggregate data sets during set operations, such as MINUS and INTERSECT, by generating hash indices and storing records in a packed form to reduce computational space and time requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a sort-based approach is used to perform set operations, then the sets can be combined according to the particular set operation, but the sorting contributes substantially to the latency of the set operation
Solution Approach 1:
The patent changes the fundamental parameter of how sets are processed by replacing the sort-based sequential scanning approach with a hash-based approach. Instead of sorting sets A and B and scanning them sequentially, the patent applies a hash function to elements of both sets, storing hashed elements in hash tables. This parameter change from sorting to hashing eliminates the time-consuming sorting step while maintaining the ability to perform set operations (intersection, difference, union) by comparing hashed values in constant time.
2Device complexity
If sets are pre-sorted in the expected order, then the sets do not need to be sorted as part of the set operation, but this requires additional storage space and preprocessing
Solution Approach 1:
The patent creates a copy of the set elements in a transformed form (hashed values) stored in hash tables. Instead of requiring the original sets to be pre-sorted in memory, the patent copies elements into hash table structures where they can be efficiently accessed and compared. This copying approach eliminates the need for pre-sorting while using manageable storage space for the hash tables, which only store the essential keyed information needed for set operations.
Data Source
AI summary
Computer-implemented techniques for hash-based set operations. In some embodiments, the techniques are implemented in a computer database management system to improve the computational space or time efficiency of executing database query language statements that contain one or more set operations. With the hash-based techniques, duplicate record elimination and aggregation of the component query result sets is not required before combining the sets in a set operation as the set operation itself performs aggregation on the records. As a result, the computational efficiency of performing the set operation is improved over a sort-based approach where a component query result set is not pre-sorted.