Hash-Based Set Similarity Derivation With Duplicate Elimination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining similarity degree between sets require excessive computation of hash values due to the need for multiple hash functions, leading to inefficient processing.
Innovation Solution
A system and method that eliminates duplicate calculations of hash functions for equivalent hash values, using a predetermined hash function with the widest range to reduce the number of hash value computations, and identifies minimum hash values for each set to derive similarity degree.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple hash functions are applied to each element of each set to obtain multiple minimum hash values for similarity degree determination, then the accuracy of similarity degree measurement is improved, but the computational burden increases significantly
Solution Approach 1:
The patent applies preliminary action by first computing hash values using a predetermined hash function for all elements across all sets, identifying duplicate elements before applying other hash functions. This preliminary identification and elimination of duplicates reduces the number of subsequent hash computations needed, thereby lowering the computational burden while preserving the ability to calculate multiple minimum hash values for accurate similarity measurement.
2Reliability
If multiple hash functions are applied to each element of each set to obtain multiple minimum hash values, then the reliability of similarity degree determination is improved, but the processing time increases
Solution Approach 1:
The patent performs preliminary computation of hash values using a predetermined hash function for all elements before applying other hash functions. By identifying and eliminating duplicate elements in this preliminary stage, the system reduces the number of elements that require subsequent hash computations, thereby reducing processing time while maintaining the reliability of similarity degree determination through multiple minimum hash values.
Solution Approach 2:
The patent segments the computation process into distinct stages: first computing hash values with a predetermined hash function, then identifying duplicates, and finally applying other hash functions only to non-duplicate elements. This segmentation of the computation process allows for optimized resource allocation and reduced processing time while preserving measurement reliability.
3Measurement precision
If hash value calculations are performed for all elements of all sets using multiple hash functions, then complete similarity analysis is achieved, but redundant calculations occur for equivalent hash values
Solution Approach 1:
The patent applies preliminary action by computing hash values using a predetermined hash function for all elements and identifying duplicates before applying other hash functions. This preliminary identification eliminates redundant calculations of equivalent hash values, reducing energy consumption while maintaining complete similarity analysis through subsequent computation of minimum hash values for non-duplicate elements.
Data Source
AI summary
The hash value calculation means 2, when obtaining multiple hash values obtained by applying multiple hash functions to individual elements of each set in multiple sets, obtains the multiple hash values for individual elements of each set, eliminating duplicate calculations of each hash function other than a predetermined hash function among the multiple hash functions with respect to multiple elements whose hash values obtained by the predetermined hash function match and which are equivalent. The minimum hash value identification means 3 identifies minimum hash value which is the minimum value of hash values for each individual hash function, for each set in the multiple sets. The similarity degree derivation means 4 derives, with respect to one or more pairs of sets obtained from the multiple sets, similarity degree of two sets that form a pair based on minimum hash values corresponding to individual hash functions.