Hash Table Segmentation for Efficient Data Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data retrieval methods using hash tables face efficiency degradation due to overlapping hash values, especially for large data sets, and require significant time for calculating perfect hash functions, leading to slow data storage and retrieval speeds.
Innovation Solution
A data storage program that divides a large data set into smaller sets, determines multiple hash functions for each set, and uses near-perfect hash functions to store data efficiently, reducing the number of hash functions needed and improving storage and retrieval speeds by utilizing the CPU cache memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a perfect hash function is used to guarantee unique hash values for all data, then data retrieval efficiency is maximized, but the calculation cost increases exponentially with data set size
Solution Approach 1:
The patent divides a large data set into multiple small-scale data sets, and calculates a perfect hash function for each small-scale data set individually. This segmentation approach reduces the calculation complexity from exponential growth with respect to the entire large data set to polynomial growth with respect to each small-scale data set, making the calculation feasible while maintaining perfect hash properties within each segment.
2Speed
If a hash table is used for data retrieval, then retrieval speed is improved, but hash value overlaps cause efficiency degradation
Solution Approach 1:
The patent segments the large data set into multiple small-scale data sets and assigns a dedicated perfect hash function to each segment. This ensures that within each hash table segment, no hash value overlaps occur, eliminating the efficiency degradation problem while maintaining fast retrieval speeds through hash table structure.
3Manufacturing precision
If a large data set is processed as a whole to determine a perfect hash function, then optimal hash distribution is achieved, but the calculation becomes computationally infeasible
Solution Approach 1:
The patent divides the large data set into multiple small-scale data sets that can be processed independently and in parallel. This segmentation transforms an computationally infeasible single large-scale calculation into multiple manageable small-scale calculations, significantly improving calculation throughput while maintaining optimal hash value distribution within each segment through perfect hash functions.
Solution Approach 2:
The patent performs preliminary actions by first dividing the large data set into small-scale data sets before calculating hash functions. This preliminary segmentation enables subsequent parallel processing and reduces the overall calculation burden, making the determination of perfect hash functions feasible for large-scale data while maintaining optimal distribution properties.
Data Source
AI summary
A computer executes a process including dividing a data set into a plurality of data sets, determining hash functions for the data sets, producing hash coefficient value information for specifying the hash functions and correspondence information between the hash coefficient values and the data sets, and producing hash information for the data sets. In the hash function determination, a hash value is calculated using a candidate hash function based on keys of the data of the data set, and the candidate hash function is determined as the hash function of the data set with regard to which it is decided that all data can be stored into a first address based on the hash value or a second address contiguous to the first address. In the hash information production, the hash information for the data set is produced by storing the data and keys into the first or second address.