Multistage Hash Table for Storage Controller Write Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems face inefficiencies in eliminating duplicate data due to the increased storage capacity requirements of hash tables, leading to degraded write performance caused by numerous I/O processes when searching for hash values.
Innovation Solution
A storage system with a hash table configuration that includes multiple stages of tables with varying entry sizes, allowing for efficient classification and storage of hash values, reducing the search range and number of I/O processes by using a second-stage hash value to determine the appropriate stage for searching, thereby improving write performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a multistage configuration with multiple tables is used to store hash values, then the storage capacity to store the hash table increases, but the number of I/O processes increases and write performance degrades
Solution Approach 1:
The hash table is segmented into multiple stages with different numbers of entries. The first stage table has a first number of entries and the second stage table has a second number of entries, allowing the system to store a large number of hash values while reducing the search range in each stage, thereby reducing I/O processes and improving write performance
Solution Approach 2:
The patent introduces a hierarchical dimension to the hash table structure by adding a second stage table. This dimensional change allows the system to organize hash values in multiple levels, where the first stage provides broad classification and the second stage provides detailed storage, reducing the need to scan the entire hash table and thus reducing I/O operations
2Quantity of substance
If the hash table is stored in a large-capacity storage device, then the storage capacity is sufficient, but many I/O processes occur between the storage controller and the storage device, degrading write speed
Solution Approach 1:
The hash table is divided into multiple stages with different entry counts. The first stage table with fewer entries provides quick classification, while the second stage table with more entries provides comprehensive storage. This segmentation reduces the average search range and minimizes I/O processes between the storage controller and the storage device, thereby improving write speed while maintaining sufficient storage capacity
Solution Approach 2:
The first stage table performs preliminary classification of hash values before they are stored in the second stage table. This preliminary action reduces the search space for subsequent hash value lookups, minimizing the number of I/O operations needed and improving overall write performance
Data Source
AI summary
According to one embodiment, a storage controller includes a dividing unit, a duplication manager, and a duplication determination unit. The dividing unit divides data specified in a write request from a host computer into a plurality of chunks. The duplication manager preferentially stores a first hash value of a first chunk in a first table in a hash table in association with the first chunk when the first chunk is written to a storage device. The hash table includes a second table having more entries than the first table. The duplication determination unit first searches the first table for a third hash value matching a second hash value of a second chunk when the second hash value has been calculated.