Occurrence Count Storage for Hash-Based Frequency Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently estimating the frequently occurring portion of a set of values, particularly in scenarios where the set of values is large and dynamic, leading to high computational and storage requirements.
Innovation Solution
The proposed solution involves an apparatus with hash circuitry to generate hash values from a set of values, signature storage circuitry to store signature data indicative of a subset of hash values, and calculation circuitry to estimate the frequently occurring portion based on occurrence counts meeting a predetermined condition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all hash values from a large set are stored to accurately estimate the frequently occurring portion, then measurement precision is improved, but storage requirements and device complexity increase significantly
Solution Approach 1:
The patent segments the large set of hash values into a smaller representative subset for storage and analysis. Instead of storing all hash values, the system maintains a manageable subset that still provides accurate statistical estimates of the frequently occurring portion, thus reducing storage requirements while preserving measurement precision.
Solution Approach 2:
The patent extracts only the essential information needed for estimation by storing occurrence counts for hash values that meet a predetermined condition (indicating frequent occurrence), rather than storing complete hash value information. This extraction approach reduces the quantity of stored data while maintaining the ability to accurately estimate the frequently occurring portion.
2Measurement precision
If a large subset of hash values is maintained to improve estimation accuracy, then measurement precision is improved, but computational requirements and processing time increase
Solution Approach 1:
The patent applies local quality by focusing computational resources on storing and processing only those hash values that meet a predetermined condition (i.e., frequently occurring values). Rather than uniformly processing all hash values, the system selectively maintains information about locally significant elements, improving processing efficiency while preserving estimation accuracy.
Solution Approach 2:
The patent performs partial action by storing occurrence counts only for hash values that satisfy a predetermined condition, rather than processing and storing information for all hash values. This partial processing approach reduces computational requirements and improves productivity while still providing accurate estimates of the frequently occurring portion.
3Quantity of substance
If the subset of stored hash values is dynamically adjusted to reduce storage requirements, then device complexity increases due to dynamic management, but storage efficiency improves
Solution Approach 1:
The patent implements dynamics by allowing the subset of stored hash values to be dynamically adjusted based on incoming data and predetermined conditions. The system can add new hash values that meet the condition and remove or update existing ones, enabling adaptive storage management that improves storage efficiency while handling the complexity through systematic dynamic updates.
Solution Approach 2:
The patent applies self-service by using the occurrence count mechanism to automatically identify and retain only those hash values that meet the predetermined condition for frequent occurrence. The system self-manages the subset composition based on the data itself, reducing the need for external complex management while improving storage efficiency through automatic filtering and retention of significant values.
Data Source
AI summary
There is provided an apparatus comprising hash circuitry to generate hash values from a set of values, and signature storage circuitry to store signature data indicative of a subset of the hash values. The signature storage circuitry is responsive to receipt of a hash value that falls within a range defined based on the subset, to retain the hash value in the subset and to discard a member of the subset. The signature storage circuitry is configured, when the hash value does not fall within the dynamically varying range, to discard the hash value. The signature storage circuitry is configured to store an occurrence count indicative of repeat occurrences of each stored hash value. The apparatus is provided with calculation circuitry to estimate a frequently occurring portion of the values based on stored hash values for which the occurrence count meets a condition.