Direct Mapped Hash Table Persistence in Non-Volatile Memory

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional methods for persisting data structures from volatile to non-volatile memory in information processing systems are inefficient, leading to substantial computational and memory resource wastage, which undermines system performance.

Innovation Solution

The implementation of a direct mapped hash table that splits buckets into chunks and allocates them optimally in non-volatile memory, reducing the number of blocks written and optimizing amortization, thereby improving storage system performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional approaches are used for persisting data structures from volatile to non-volatile memory, then data persistence is achieved, but computational and memory resources are wasted substantially

Engineering Contradiction:

Improvedata persistenceVSAvoidcomputational resource wastage

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent divides the hash table into multiple shards, where each shard is a separate data structure that can be independently persisted to non-volatile memory. This segmentation allows the system to persist only the necessary portions of the hash table, reducing computational overhead and memory resource wastage while maintaining data persistence reliability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements selective persistence where different parts of the hash table (shards) are persisted based on their specific requirements and usage patterns. This local quality approach allows the system to optimize resource consumption by persisting only those shards that are currently in use or require persistence, rather than persisting the entire hash table structure.

Inventive Principle:

Principle #3Local quality

2Reliability

If conventional approaches are used for persisting data structures from volatile to non-volatile memory, then data persistence is achieved, but memory resources are wasted substantially

Engineering Contradiction:

Improvedata persistenceVSAvoidmemory resource consumption

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent segments the hash table into multiple independent shards, each of which can be persisted to non-volatile memory separately. This segmentation enables the system to allocate memory resources more efficiently by only allocating and persisting the necessary shards, thereby reducing overall memory consumption while maintaining data persistence capability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by implementing selective persistence at the shard level. Each shard is persisted based on its specific needs and usage patterns, allowing the system to optimize memory resource allocation by only persisting shards that are currently required, rather than persisting the entire hash table structure.

Inventive Principle:

Principle #3Local quality

3Quantity of substance

If buckets are split into chunks and allocated optimally in non-volatile memory, then space efficiency is improved, but device complexity increases

Engineering Contradiction:

Improvespace efficiencyVSAvoidhash table structure complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent segments each bucket into multiple chunks that can be independently allocated to different shards. This segmentation, combined with the shard-based architecture, allows for flexible and efficient space utilization in non-volatile memory while managing complexity through a systematic division of the hash table structure.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements a dynamic allocation mechanism where chunks from buckets can be assigned to different shards based on current system needs and usage patterns. This dynamic approach allows the system to optimize space efficiency by allocating chunks to the most appropriate shards, while the complexity is managed through automated allocation algorithms.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10831735B2Processing device configured for efficient generation of a direct mapped hash table persisted to non-volatile block memory

Publication Date: 2020.11.10 EMC IP HLDG CO LLC

AI summary

An apparatus in one embodiment comprises at least one processing device. The processing device is configured to generate a hash entry of a hash table implementing a key-value store that comprises a plurality of buckets associated with corresponding keys where a given bucket comprises a bucket index having two or more entries for storage of values that hash to a key associated with the given bucket. The processing device is also configured to allocate blocks in non-volatile memory, to split the given bucket into chunks each comprising at least one entry of the bucket index, and to persist the hash table to the non-volatile memory utilizing a direct mapping of the chunks of the given bucket to respective designated block offset positions in the allocated blocks. Persisting the hash table comprises storing first and second chunks of the given bucket in first and second allocated blocks.