Hash Table Memory Optimization Using Binomial Distribution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current hash table implementations face inefficiencies in memory allocation, particularly with linked lists, which result in high overhead and memory fragmentation due to dynamic size adjustments and the need for frequent array resizing, especially when dealing with large numbers of entries.

Innovation Solution

The method organizes hash table data storage using arrays instead of linked lists, predicting array sizes using the binomial distribution method to ensure efficient memory usage and reduce fragmentation, allowing for direct memory access to optimize CPU performance and memory allocation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If linked lists are used to store hash table groups, then dynamic allocation of memory allows easy size changes, but each element requires additional link references increasing memory overhead

Engineering Contradiction:

Improvedynamic memory allocationVSAvoidmemory overhead

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent applies preliminary action by predicting the final size of each hash group using binomial distribution calculations before inserting elements. This allows pre-allocation of arrays with the correct size, eliminating the need for dynamic resizing and reducing memory overhead from link references while maintaining adaptability through accurate size prediction.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If arrays are used instead of linked lists to store hash table groups, then additional link references are eliminated reducing memory overhead, but the size of the array is not known previously requiring larger allocation or resizing

Engineering Contradiction:

Improvememory overheadVSAvoidarray size determination

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent resolves the array size determination problem by performing preliminary calculations using binomial distribution to predict the exact size of each hash group before element insertion. This eliminates the complexity of dynamic resizing while ensuring arrays are allocated with the precise size needed, reducing memory overhead without requiring complex size determination mechanisms during runtime.

Inventive Principle:

Principle #10Preliminary action

3Quantity of substance

If arrays are resized frequently to match actual group sizes, then memory usage is optimized, but copying elements leads to memory fragmentation and inefficiency

Engineering Contradiction:

Improvememory usageVSAvoidelement copying efficiency

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The patent eliminates element copying and memory fragmentation by performing preliminary size prediction using binomial distribution before element insertion. This allows arrays to be allocated with the correct size from the beginning, eliminating the need for frequent resizing operations and the associated element copying that causes memory fragmentation, thus optimizing both memory usage and productivity.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11080251B1Optimization of memory usage while creating hash table

Publication Date: 2021.08.03 COMODO SECURITY SOLUTIONS INC

AI summary

There is provided a method to optimize memory usage in hash tables by organizing data storage in the hash table with arrays instead of linked lists where sizes of arrays can be predicted with high accuracy. The method has certain conditions and range of applicability, including where hash function of key provides uniform distribution of values. The method is more applicable if the number of entries is more than 60,000.