System and method for efficiently expanding key-value hash tables

Abstract

Embodiments of the present invention generally relate to systems and methods for efficiently expanding key-value hash tables. A device for managing key-value KV data in a non-volatile memory includes a controller and a volatile memory storing a hash table. The controller: generates a first key from a KV pair; reads a first entry of the hash table based on the first key; reads a first page containing a set of KV hash entries based on the first entry, each of the set of KV hash entries containing a location of the non-volatile memory; determines whether the number of entries in the first page reaches a predetermined number; and in response to determining that the number of entries in the first page reaches the predetermined number, stores the KV data corresponding to the KV pair in a location of the non-volatile memory, writes the KV hash entry containing the location of the non-volatile memory, and writes the location of the KV hash entry in a second entry of the hash table.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to systems and methods for operating key-value data storage devices, and more particularly to systems and methods for managing key-value data in data storage devices using key-value hash tables without duplicating the hash tables. Background Art

[0002] A key-value store uses an input user key to identify a data block via a key-to-physical address mapping. A hash table can be used for this address mapping. The hash table can be maintained in local memory, such as volatile dynamic random access memory (DRAM). Given the limited capacity of DRAM, it is still desirable to improve the efficient management of the hash table. Summary of the Invention

[0003] Embodiments of the present invention relate to systems and methods for managing key-value data in a data storage device and expanding the key space of a key-value hash table without doubling the size of the hash table.

[0004] According to a specific aspect, an embodiment provides a device for managing key-value (KV) data in a non-volatile memory, comprising a controller and a volatile memory storing a hash table. The controller may be configured to generate a first key from a key-value (KV) pair. The controller may be configured to read a first entry of the hash table based on the first key. The controller may be configured to read a first page containing a set of KV hash entries based on the first entry of the hash table, each of the set of KV hash entries containing a location of the non-volatile memory. The controller may be configured to determine whether the number of entries in the first page reaches a predetermined number. The controller may be configured to, in response to determining that the number of entries in the first page reaches the predetermined number, store the KV data corresponding to the KV pair in a first location of the non-volatile memory, write the first KV hash entry containing the first location of the non-volatile memory, and write the location of the first KV hash entry in a second entry of the hash table. The controller may be configured to, in response to determining that the number of entries of the first page has not reached the predetermined number, store the KV data corresponding to the KV pair in a second location of the non-volatile memory and add a new KV hash entry containing the second location of the non-volatile memory to the first page. In some embodiments, each of the first entry and the second entry of the hash table may contain the location of the non-volatile memory.

[0005] According to other aspects, an embodiment provides a method for managing key-value (KV) data in a non-volatile memory. The method may include: generating, by a processor, a first key from a key-value (KV) pair. The method may include: reading, by the processor, a first entry of a hash table stored in a volatile memory based on the first key. The method may include: reading, by the processor, a first page containing a set of KV hash entries based on the first entry of the hash table, each of the set of KV hash entries containing a location of a non-volatile memory. The method may include: determining, by the processor, whether the number of entries of the first page reaches a predetermined number. The method may include: in response to determining that the number of entries of the first page reaches the predetermined number, storing, by the processor, the KV data corresponding to the KV pair in a first location of the non-volatile memory, writing the first KV hash entry containing the first location of the non-volatile memory, and writing the location of the first KV hash entry in a second entry of the hash table. The method may include: in response to determining that the number of entries of the first page does not reach the predetermined number, storing, by the processor, the KV data corresponding to the KV pair in a second location of the non-volatile memory, and adding a new KV hash entry containing the second location of the non-volatile memory to the first page. In some embodiments, each of the first entry and the second entry of the hash table contains the location of the non-volatile memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] These and other aspects and features of embodiments of the present invention will become apparent to those skilled in the art upon review of the following description of specific embodiments in conjunction with the accompanying drawings, in which:

[0007] Figure 1 illustrates the distribution of the number of entries in a page (or bucket) of a hash table (e.g., a metadata map) according to some embodiments;

[0008] Figure 2 is a block diagram illustrating an example flash memory system according to some embodiments;

[0009] Figure 3A and Figure 3B is a block diagram illustrating an example key-to-flash memory mapping arrangement using a hash table according to some embodiments;

[0010] Figure 4 Illustrates an example process for performing redirection in a hash table according to some embodiments;

[0011] Figure 5A 、 Figure 5B and Figure 5CIllustrate another example process for performing redirection in a hash table according to some embodiments;

[0012] Figure 6A illustrates example pseudo-code for calculating a redirected address using a hash table according to some embodiments;

[0013] Figure 6B is a flowchart illustrating an example method for calculating a redirected address using a hash table according to some embodiments;

[0014] Figure 7 is a flowchart illustrating an example method for storing key-value (KV) data in non-volatile memory using a hash table according to some embodiments; and

[0015] Figure 8 is a flowchart illustrating an example method for reading KV data in non-volatile memory using a hash table, according to some embodiments. DETAILED DESCRIPTION

[0016] According to certain aspects, embodiments in this disclosure relate to techniques for managing key-value data in a data storage device and expanding the key space of a key-value hash table without doubling the size of the hash table.

[0017] Conventional hash tables are typically powers of two (2). This can create memory challenges for very large hash tables. Various metrics are commonly used to determine when to double the table size of a hash table. There are many different key-value (KV) designs that utilize some form of hash table. In flash-based native KV-FTLs (key-value flash translation layers), there is a problem that resources on SSDs (solid-state drives) are limited and at some point the hash table size may not be doubled anymore.

[0018] For example, a fixed length key may be generated from a user key (using a hash function) and used to describe a key-value pair. A key may be variably split into an upper key or index (e.g., an index into a hash table) and a lower key or remainder. This upper key may be used to index a hash table (an array of hash buckets). In some embodiments, a metadata page may be a bucket of the hash table. Each entry in the hash table may point to a flash location in a metadata page containing a table of entries. Initially, there may be only one metadata page (the initial metadata page), and therefore all keys may be contained in the metadata page. As additional key values ​​are written, the initial page may eventually be split into two (2), with the least significant bit (LSB) of the upper key indicating which page the corresponding entry goes into. The 1 bit may no longer be stored in the table. Over time, the metadata page may be split again and again, with more bits used as indexes. In extreme cases, the size of the hash table may become a limitation, and the ability to simply double the size of the hash table may become an issue.

[0019] Figure 1 The diagram illustrates the distribution of the frequency of occurrence of the number of entries in a page (or bucket) of a hash table (e.g., metadata map) according to some embodiments. Assuming a good hash function, the number of entries in a page can have a normal distribution. For example, in most cases, the number of entries in a metadata page does not exceed the page limit, e.g. Figure 1 However, at the limit, such as when the number of entries in the metadata page reaches the page limit, it may only take one entry exceeding the page limit to force the hash table to double. In other words, the tail of the distribution of the number of entries (e.g., Figure 1 The portion 101 in the hash table may trigger a doubling of the entire hash table. To avoid this problem, more DRAM and / or secondary data structures may be added. However, secondary data structures may reduce performance or limit the number of objects for a given DRAM capacity.

[0020] To address this issue, according to certain aspects, embodiments in the present disclosure relate to techniques for using address redirection to expand a hash table (e.g., a metadata map) beyond a smaller percentage of a size limit that is a given power of 2 to avoid doubling the size of the table according to existing solutions when the size limit is reached. In some embodiments, a value can be guaranteed to be retrieved in two (2) DRAM reads and one (1) flash read, and the worst-case buckets can be prevented from limiting the average KV capacity. In some embodiments, address redirection can be used for a smaller percentage of buckets to allow O(2) complexity for those buckets (e.g., 2 DRAM reads and 1 flash read), while maintaining O(1) for the remainder (e.g., 1 DRAM read and 1 flash read), thereby allowing the hash table to have significantly more capacity without doubling resources.

[0021] In some embodiments, the key-value storage system may include a metadata map that is a KV hash table. Each entry of the metadata map may point to a flash location of a metadata page containing a table of KV hash entries. The metadata map may be stored in a memory (e.g., DRAM). The KV hash table may initially place all keys in an initial metadata page (page 0). Once the initial metadata page is full, the initial metadata page may be split into two based on the least significant bit (LSB) in the upper key, so that there are two (2) metadata pages—the initial metadata page and the first metadata page (page 0 and page 1) and the length of the upper key index is equal to 1 (upper key index length = 1). When either of the two metadata pages is full, the full page may be split. For example, if page 1 is full, page 1 may be split into page 1 and page 3, so that the length of the upper key index is equal to 2 (upper key index length = 2). If an attempt is made to read page 2, which does not yet exist, a lookup of the metadata map may return an invalid location (e.g., address 0, or some other reserved or unused value). If this occurs, an attempt can be made to read page 0, which will succeed. This hash table splitting (or growing) process can continue as needed as more objects or entries are written. In some embodiments, this splitting (or growing) process can be implemented by causing a processor (e.g., a CPU) to execute program instructions. In some embodiments, this splitting (or growing) process can be implemented in hardware or firmware.

[0022] In some embodiments, the metadata map may be in powers of two (e.g., 2 30 ) area to allow for overflow of the KV hash table. In some embodiments, the address space defined by the entries of the metadata map may include a cache address space, a redirect address space, and a flash address space. In some embodiments, the address in the power of two area of ​​the metadata map may be considered as an indirect value pointing to the beginning of the redirect address space and an offset value from the beginning of the redirect address space (rather than pointing to an address in the flash space). The value found at the offset address can be used as a reference to two (2) entries in the overflow area of ​​the metadata map. The redirected address in the flash address space can be determined based on the two entries in the overflow area of ​​the metadata map. In some embodiments, the cache can be managed in a similar manner.

[0023] According to certain aspects, embodiments of the present disclosure relate to a device for managing key-value (KV) data in a non-volatile memory, comprising a controller and a volatile memory storing a hash table. The controller may be configured to generate a first key from a key-value (KV) pair. The controller may be configured to read a first entry of the hash table based on the first key. The controller may be configured to read a first page containing a set of KV hash entries based on the first entry of the hash table, each of the set of KV hash entries containing a location of the non-volatile memory. The controller may be configured to determine whether the number of entries in the first page reaches a predetermined number. The controller may be configured to, in response to determining that the number of entries in the first page reaches a predetermined number, store the KV data corresponding to the KV pair in a first location of the non-volatile memory, write the first KV hash entry containing the first location of the non-volatile memory, and write the location of the first KV hash entry in the second entry of the hash table. The controller may be configured to, in response to determining that the number of entries in the first page has not reached a predetermined number, store the KV data corresponding to the KV pair in a second location of the non-volatile memory and add a new KV hash entry containing the second location of the non-volatile memory to the first page. In some embodiments, each of the first entry and the second entry of the hash table may contain a location of the non-volatile memory.

[0024] According to certain aspects, embodiments of the present disclosure relate to a method for managing key-value (KV) data in non-volatile memory. The method may include, by a processor, generating a first key from a key-value (KV) pair. The method may include, by the processor, reading a first entry of a hash table stored in volatile memory based on the first key. The method may include, by the processor, reading a first page containing a set of KV hash entries based on the first entry of the hash table, each of the set of KV hash entries containing a location in the non-volatile memory. The method may include, by the processor, determining whether the number of entries in the first page reaches a predetermined number. The method may include, in response to determining that the number of entries in the first page reaches the predetermined number, storing, by the processor, KV data corresponding to the KV pair in a first location in the non-volatile memory, writing a first KV hash entry containing the first location in the non-volatile memory, and writing the location of the first KV hash entry in a second entry of the hash table. The method may include, in response to determining that the number of entries in the first page does not reach the predetermined number, storing, by the processor, the KV data corresponding to the KV pair in a second location in the non-volatile memory, and adding a new KV hash entry containing the second location in the non-volatile memory to the first page. In some embodiments, each of the first entry and the second entry of the hash table contains a location of non-volatile memory.

[0025] Embodiments of the present disclosure have at least the following advantages and benefits.

[0026] Embodiments in the present disclosure may provide useful techniques for preventing a hash table (e.g., a metadata map) from growing when an initial metadata page exceeds its limit due to an uneven number of keys in the metadata page. For example, an additional overflow area of ​​the metadata map outside of the power-of-two area may be used to avoid doubling the size of the metadata map for each additional bit of the encoded upper key. These techniques may enable a hash table to have increased capacity without doubling the resources. These techniques may enable a higher number of keys to be stored at a minimal increased resource cost, as well as increase the overall efficiency of the resources used, thereby increasing the value of the solution to the customer.

[0027] Figure 2 is a block diagram illustrating an example flash memory system in accordance with some embodiments.

[0028] refer to Figure 2 The flash memory system may include a solid-state drive (SSD) 200, which is a storage device that can be used as a primary storage device for an information processing device (e.g., a host computer). SSD 200 may be incorporated into the information processing device or may be connected to the information processing device via a cable or network. For example, SSD 200 includes a controller 220 and flash memory 280 (e.g., NAND-type flash memory), which is a non-volatile memory. SSD 200 may include random access memory (RAM), which is a volatile memory, such as DRAM (dynamic random access memory) 210. In some embodiments, controller 220 may include random access memory, such as SRAM (static random access memory). For example, random access memory (e.g., DRAM 210) has a read buffer, which is a buffer area for temporarily storing data read from flash memory 280; a write buffer, which is a buffer area for temporarily storing data written to flash memory 280; and a buffer for garbage collection. In some embodiments, controller 220 may include DRAM 210.

[0029] In some embodiments, the flash memory 280 may include a memory cell array including a plurality of flash memory blocks (e.g., NAND blocks) 282-1 to 282-m. Each of the blocks 282-1 to 282-m may be used as an erase unit. Each of the blocks 282-1 to 282-m includes a plurality of physical pages. In some embodiments, data reading and data writing are performed on a page basis in the flash memory 280, and data erasure is performed on a block basis.

[0030] In some embodiments, the controller 220 may be a memory controller configured to control the flash memory 280. For example, the controller 220 includes a processor (e.g., a CPU) 250, a flash memory interface 240, and a DRAM interface 230, all of which may be interconnected via a bus 228. The DRAM interface 230 may function as a DRAM controller configured to control access to the DRAM 210. The flash memory interface 240 may function as a flash memory control circuit (e.g., a NAND control circuit) configured to control the flash memory 280 (e.g., a NAND-type flash memory).

[0031] The processor 250 may be configured to control the flash memory interface 240 and the DRAM interface 230. The processor 250 may be configured to perform various processes by executing a control program (e.g., firmware) stored in, for example, a ROM (not shown). In some embodiments, the processor 250 may execute a command control 260 to perform command processing for processing various commands received from an information processing device (e.g., a host computer).

[0032] The processor 250 may be configured to function as a flash translation layer (FTL) 270 to perform data management and block management of the flash memory 280. The FTL 270 may include a lookup table control 272, a garbage collection control 274, a wear leveling control 276, and a flash memory control 278. Data management may include managing mapping information indicating a correspondence between a user key and a physical address of the flash memory 280. In some embodiments, the lookup table control 272 may use an address translation table (logical / physical address translation table) to perform mapping management between each logical block address (LBA) or user key and each physical address. The lookup table control 272 may perform hashing of the user key to perform a hash table (e.g., Figure 3A The key-value data is managed by indexing the metadata map 330 in the SSD. The garbage collection control 274 can perform garbage collection (GC), which is a process performed to generate free blocks as data write destination blocks. The wear leveling control 276 can perform wear leveling, which is a process of balancing the number of block erases so that the failure probability of the SSD 3 can be reduced by preventing the occurrence of blocks with a large number of erases. The flash memory control 278 can perform control of the flash memory interface 240.

[0033] Figure 3A and Figure 3B is a block diagram illustrating an example flash memory system using a hash table in accordance with some embodiments.

[0034] refer to Figure 3A , a processor of a flash memory system (e.g., Figure 2The processor 250 in FIG. 2 may generate a fixed length key 310 from a variable length user key 302 of a key-value pair 301 including a user key 302 and a corresponding value 305 (which is not part of the hash value). A hash function (not shown) may be used to generate the fixed length key 310 from the variable length user key 302. The key 310 may be used as a reference for the key-value pair 301. The fixed length key 310 may be variably split into an upper key or index 312 (e.g., an index into a hash table) and a lower key or remainder 314. This upper key may be used to index a hash table (hash bucket array). For example, a 32-bit upper key 312 may be masked by a mask 320 to generate an index into a hash table (e.g., Figure 3A In some embodiments, the mask may be defined using the upper key index length value (UpperKeyIndexLen) as follows:

[0035] Mask = 2 UpperKeyIndexLen -1 (Equation 1)

[0036] In some embodiments, metadata map 330 may be stored in a memory (e.g., Figure 2 The metadata map 330 includes a plurality of mapping entries. In some embodiments, each entry is 4 bytes (B) long to describe the non-volatile memory of the metadata page (e.g., Figure 2 For example, in the flash memory 280) or the in-memory cache location. Figure 3A 312 by mask 320, and entry 334 may contain a 4-byte address of nonvolatile memory indicating the location of metadata page 1 (344). Similarly, entry 332 may contain a 4-byte address of nonvolatile memory indicating the location of metadata page 0 (342). Assuming a 1KB size per metadata page, the 4-byte address contained in the metadata map entry may define 2 equivalent terabytes of metadata in flash memory. 32 However, since this address space can also include over-provisioning, it is more practical to store 2TB of metadata. For example, 2TB of metadata may be required to store 128TB of user key-value pairs.

[0037] In some embodiments, the metadata page (e.g., Figure 3A342 and 344 in the metadata map 330) can be buckets of a hash table, such as metadata map 330. The metadata page can contain a table of KV hash entries, each of which can contain an address indicating a non-volatile memory address of the location of a value corresponding to a key. For example, metadata page 1 (344) can contain KV hash entry 0 (391), KV hash entry 1 (392), etc., and KV hash entry 392 can contain the address and length of value 305 stored in non-volatile memory.

[0038] refer to Figure 3B , the KV hash entry may be a KV hash entry 350 for a small value or a KV hash entry 360 for a large value. The KV hash entry 350 for a small value may include the remaining unmasked bits of the 32-bit upper key 351, the 64-bit lower key 352, a flash address 353 having 44 bits or less, and a 14-bit length 354. The KV hash entry 360 for a large value may include the remaining unmasked bits of the 32-bit upper key 361, the 64-bit lower key 362, a reserved (RSVD) field 363, a 3-bit page size (PS) 364, a 32-bit metadata page number 365, and a predefined constant 366 (e.g., "3FFF"). In some embodiments, the three (3) bits of the page size (PS) 364 may contain values ​​of 0, 1, 2, 3, 4 to indicate page sizes of 4KB, 8KB, 16KB, 32KB, and 64KB, respectively. In some embodiments, a metadata page may contain a segment descriptor 370 or a segment indirection for large values ​​380. The segment descriptor 370 may include a flash address 371 having 44 bits or less and a 14-bit length 372. The segment indirection for large values ​​380 may include a 32-bit metadata page number 381. In some embodiments, all data in a metadata page may be bit packed.

[0039] In some embodiments, a key-value storage system (e.g., Figure 2 The SSD 200 in FIG. 200 may include a metadata map (eg, metadata map 330) as a KV hash table. The KV hash table may initially have all keys placed into an initial metadata page (eg, Figure 3A Once the initial metadata page is full, it can be split into two based on the least significant bit (LSB) in the upper key so that there are two (2) metadata pages—the initial metadata page and the first metadata page (e.g., Figure 3A0 (342) and page 1 (344) in the metadata map) and the length of the upper key index (UpperKeyIndexLen) is equal to 1 (UpperKeyIndexLen=1). When either of the two metadata pages fills up, the filled page can be split. For example, if page 1 fills up, then page 1 can be split into page 1 and page 3 so that the length of the upper key index is equal to 2 (UpperKeyIndexLen=2). If an attempt is made to read page 2, which does not yet exist, then the lookup of the metadata map may return an invalid location (e.g., address 0). If this happens, then an attempt can be made to read page 0, which will succeed. As more objects or entries are written, this hash table splitting (or growing) process can continue as needed. Over time, metadata pages can be split again and again, with more bits used as indexes. In extreme cases, the size of the hash table can become a limitation, and the ability to simply double the size of the hash table can become a problem. For example, if the size of the metadata map 330 is limited to less than 2 31 If the size of the metadata map is equal to the size of the entries, then when UpperKeyIndexLen=30, the system cannot double the size of the metadata map.

[0040] Figure 4 Illustrated is an example process for performing redirection in a hash table according to some embodiments.

[0041] To address the issue of doubling the size of the metadata map, in some embodiments, address redirection can be used to expand the hash table (e.g., metadata map) by a small percentage beyond the size of a power of two (2) to prevent doubling of the table at a limit (e.g., when the number of entries in a metadata page reaches the page limit). In some embodiments, this doubling point is determined by the tail of the distribution of the number of entries (e.g., Figure 1 Section 101) is defined as only one page exceeding the limit and needs to be split resulting in doubling the size of the metadata map.

[0042] refer to Figure 4 In some embodiments, the metadata map 430 as a KV hash table may have an additional overflow area 450 outside the power of two area 440 to allow for overflow of the KV hash table. For example, the size of the power of two area 440 may be 2 30 And the size of the additional overflow area 450 may be 10% of the size of the power of two area, for example, 1.3*2 26 In some embodiments, assuming a 4-byte (B) size per map entry, the address space 460 defined by the entries of the metadata map 430 may have 2 32The address space 460 may include a flash address space 462, a redirect address space 464, and an in-memory cache address space 466. For example, if the cache address space 466 has 2 20 The size of the address space is 464 (equivalent to 1GB with 1K pages) and the redirected address space has 2 27 entries, and the metadata page size is 1K, then the metadata flash address space will have an equivalent of 3.874TB (2 32 -2 27 -2 20 )*1K in size.

[0043] refer to Figure 4 In some embodiments, a fixed-length key 410 may be generated from a user key corresponding to value 405 and may be variably split into an upper key 412 and a lower key 414. For example, the size of the upper key 412 may be selected so that the size of the lower key is the fixed length of key 410 minus the size of the upper key 412. The 32-bit upper key 412 may be masked by a mask 420 to generate an index into an entry 432 of the metadata map 430. Entry 432 may contain a 4-byte address 442 indicating an address of the redirected address space 464 (rather than indicating the location of a metadata page in the flash address space 462) to perform redirection 435. The address 442 contained in entry 432 of the power-of-two region 440 of the metadata map may be considered an indirect value 468 pointing to the start address of the redirected address space 464 and an offset 469 from the start of the redirected address space 464 (rather than pointing to an address in the flash address space 462). The offset 469 may be used to determine a reference 455 to two (2) entries 438, 439 in the overflow area 450 of the metadata map (e.g., an index to the metadata map 430). In some embodiments, the reference 455 may be determined by calculating an offset 452 from the start address 451 of the overflow area 450 based on the offset 469. For example, the offset 452 may be a multiple (e.g., two) of the offset 469. The key-value storage system may determine whether the address 457 contained in the entry 438 is within the redirected address space 464. If the address 457 contained in the entry 438 is not within the redirected address space 464 and is within the flash address space 462, then the key-value storage system may read the metadata page or KV hash entry at address 457. For example, if Figure 4 As shown in , the key-value storage system can read the KV hash entry 451 containing the address of the value 405 stored in the non-volatile memory, and further read the value 405 at the address.

[0044] If the address contained in entry 438 is within the redirected address space 464, the key-value storage system may increase the length of the upper key index (UpperKeyIndexLen) in order to read the other of the two entries, such as entry 439. In other words, the redirected reference (e.g., reference 455) may point to both entries 438 and 439 of the metadata map 430, and the next key bit of key 410 may be used (by increasing UpperKeyIndexLen) to determine which of the two entries to use. In this way, the redirected address in the flash address space (e.g., address 457) may be determined based on the two entries in the metadata map. In some embodiments, redirection may be used in a similar manner to manage caches.

[0045] like Figure 4 As shown in , embodiments of the present disclosure may use address redirection to expand a hash table (e.g., metadata map 430) beyond a small percentage (e.g., 10%) of the size of a power-of-two region (region 440) to prevent doubling of the table at the limit (e.g., when the number of entries in a metadata page reaches the page limit). In some embodiments, given the user key of a key-value pair, it is guaranteed that the value can be retrieved in two (2) DRAM reads (e.g., reading two entries from the metadata map twice), and the worst-case bucket limiting the average KV capacity can be prevented. In other words, address redirection may be used for a small percentage of buckets to allow O(2) complexity for those buckets while maintaining O(1) for the rest, thereby allowing the hash table to have increased capacity without doubling resources. In some embodiments, the size of the redirected address space 464 determines the number of entries (buckets) in the power-of-two region 440 that may use address redirection.

[0046] In some embodiments, a method for managing non-volatile memory (e.g., Figure 2 A device (e.g., Figure 2 The SSD 200 in FIG. 2 may include a controller (e.g., Figure 2 220 in the controller) and store the hash table (e.g., Figure 3A and 4 volatile memory (e.g., Figure 2 DRAM 210 in ). The controller may be configured to retrieve the key-value (KV) pair (e.g., Figure 3A 301) generates a first key (e.g., Figure 3A The controller may be configured to read a first entry of the hash table based on the first key (eg, Figure 3AThe controller may be configured to read a table comprising a set of KV hash entries (e.g., Figure 3A The first page of the KV hash entry 391 and the KV hash entry 392 in (eg, Figure 3A The controller may be configured to determine a first page (e.g., Figure 3A The controller may be configured to determine whether the number of entries in the metadata page 334 in the first page has reached a predetermined number (e.g., a predefined metadata page size determined based on empirical determination or customer configuration). In response to determining that the number of entries in the first page has reached the predetermined number, the controller may be configured to store the KV data corresponding to the KV pair (e.g., stored in Figure 2 and 4 The value 405 in the flash memory 280 in the non-volatile memory is stored in a first location (e.g., Figure 2 , writes a first KV hash entry containing a first location of non-volatile memory (e.g., containing a value stored in Figure 2 and 4 405 in the flash memory 280 in the KV hash entry 451), and in the second entry of the hash table (eg, Figure 4 438 in the entry ) writes the location of the first KV hash entry (eg, Figure 4 The controller may be configured to store the KV data corresponding to the KV pair in a second location of the nonvolatile memory (e.g., value 305 is stored in address 457 in the first page) in response to determining that the number of entries of the first page has not reached a predetermined number. Figure 2 and 3A ), and to the first page (eg, Figure 3A 344 in the metadata page 344) adds a new KV hash entry containing a second location in non-volatile memory (e.g., containing Figure 2 and 3A In some embodiments, the first entry of the hash table (e.g., Figure 3A 334 in ) and the second entry (e.g., Figure 4 Each of the entries 438 in the nonvolatile memory may contain a location of the nonvolatile memory (e.g., a 4-byte address of the nonvolatile memory indicating Figure 3A The location of metadata page 1(344) in or Figure 4 The location of the KV hash entry 451 in ).

[0047] In some embodiments, in response to determining that the number of entries of the first page reaches a predetermined number, the controller may be configured to generate a new entry based on a second entry of the hash table (eg, Figure 4 The redirect address is calculated based on the index of the entry 438 in Figure 4 ), and in the third entry of the hash table (e.g., Figure 4 For example, the controller may calculate redirect address 442 by adding start address 468 to offset 469, such that offset 469 is the start address or index of overflow area 450 of entry 438 (e.g., Figure 4 The redirect address (e.g., Figure 4 The redirection address 442 in the non-volatile memory may be in the address space of the non-volatile memory (e.g., Figure 4 The redirected address space (e.g., Figure 4 within the redirection address space 464).

[0048] In some embodiments, the controller may be configured to generate a first key from the KV pair (eg, Figure 4 The controller may be configured to read a third entry of the hash table based on the first key (eg, Figure 4 The controller may be configured to select the metadata entry 432 of the metadata map 430 based on the third entry of the hash table (e.g., Figure 4 432 in the hash table) and calculate the index of the second entry of the hash table (e.g., Figure 4 The controller may be configured to determine the address or index 455 in the hash table based on the second entry (e.g., Figure 4 438 in the KV Hash entry) and read the first KV hash entry (eg, Figure 4 The controller may be configured to read the KV data corresponding to the KV pair (e.g., stored in Figure 2 and 4 405 in the flash memory 280 in the calculation of the second entry of the hash table (eg, Figure 4 438) in the index (e.g., Figure 4 455), the controller may be configured to determine the address contained in the third entry (e.g., Figure 4 The address 442 in the entry 432 in the address space of the non-volatile memory (e.g., Figure 4 The redirected address space (e.g., Figure 4, and in response to determining the address contained in the third entry (e.g., Figure 4 442 in the address 442) in the redirected address space outside the address space of the non-volatile memory, and calculating the second entry of the hash table based on the address contained in the third entry (e.g., Figure 4 438) in the index (e.g., Figure 4 455 in the .

[0049] In some embodiments, the hash table may contain a first set of entries in a power-of-two address space (e.g., in Figure 4 ), and a second set of entries outside the power-of-two address space (e.g., in Figure 4 The first set of entries may include the first entry of the hash table (e.g., Figure 3A The second set of entries may include a second entry of the hash table (e.g., Figure 4 The number of entries in the second set of entries may be less than the number of entries in the first set of entries (e.g., the number of entries in the overflow region 450 is 10% of the number of entries in the power of two region 440). The controller may be configured to distribute the entries based on the tail distribution of the number of entries in the page containing the set of KV hash entries (e.g., Figure 1 The number of the second group of entries is determined by the tail part 101 in.

[0050] In some embodiments, the controller may be configured to retrieve the KV pair (e.g., Figure 3A 301) generates a first key (e.g., Figure 3A The controller may be configured to read a first entry of the hash table based on the first key (eg, Figure 3A The controller may be configured to read the first page based on the first entry of the hash table (eg, Figure 3A The controller may be configured to read the KV data corresponding to the KV pair based on the first page (e.g., stored in Figure 2 and 3A 305 in the flash memory 280).

[0051] Figure 5A 、 Figure 5B and Figure 5C Illustrated is another example process for performing redirection in a hash table according to some embodiments.

[0052] In some embodiments, when an address in the metadata map points to a redirected address space (e.g., an address contained in entry 432 points to a redirected address space 464; see Figure 4), you can use the upper keys (for example, Figure 4 The next less significant bit of the upper key 412 in (which is the lower key (e.g., Figure 4 The most significant bit (MSB) of the lower key 414 in the byte array 414) is used to select the two entries pointed to by the redirection reference (e.g., the two entries 438, 439 pointed to by the reference 455; see Figure 4 ) in one of them. Figure 5A 、 Figure 5B and Figure 5C This is described in more detail.

[0053] refer to Figure 5A In some embodiments, the metadata map 530 as a KV hash table may have an additional overflow area 550 outside the power of two area 540 to allow for overflow of the KV hash table. Figure 2 DRAM 230 in ), so the size of the power of two region 540 can be constrained by the maximum power of 2 value because the space occupied by the metadata map 530 can be constrained to be less than twice the corresponding maximum size of the power of two region 540. However, additional space is still available for the overflow region 550, which is less than the maximum size of the power of two region 540. Therefore, the mapping contained in the power of two region 540 can be expanded without doubling the size of the power of two region 540. The fixed length key 510 can be generated from the user key corresponding to the value 505 and can be variably split into an upper key 512 and a lower key 514. The upper key 512 can be masked by the mask 520 to produce an index to an entry 532 of the metadata map 530. The entry 532 can contain an index indicating the flash address space (e.g., Figure 4 The address 557 of the location of the metadata page 544 in the flash address space 462 in the non-volatile memory. The metadata page 544 may contain KV hash entry 0 (591), KV hash entry 1 (592), etc., and KV hash entry 591 may contain the address and length of the value 505 stored in non-volatile memory.

[0054] Figure 5B An example implementation of a flash memory map arrangement is shown when the flash memory system splits the KV hash entry for metadata page 1 (544). Figure 5B In some embodiments, when the length of the upper key (UpperKeyIndexLen) is N and a new hash entry (e.g., KV hash entry 2 (593)) is added, the flash memory system may add the new hash entry in the metadata page 544 (e.g., Figure 5B The KV hash entry for metadata page 1 (544) is split between the new metadata page 2 (545) (shown in Figure 5A). After the flash memory system splits the KV hash entry for metadata page 1 (544), entry 532 may contain information indicating the redirected address space (e.g., Figure 4 The flash memory system can determine the address 542 of the address of the redirection address space 464 in order to perform the redirection 535. Figure 4 Reference 555 is determined based on address 542 in a manner similar to reference 455 in . Reference 455 may be an index of the metadata map 530 that points to two (2) entries 538, 539 in the overflow area 550 of the metadata map, such that entry 538 contains the flash memory address 557 of metadata page 1 (544) and entry 539 contains the flash memory address 558 of metadata page 2 (545). In some embodiments, when splitting the KV hash entries of metadata page 1 (544), the flash memory system may determine whether each KV hash entry of metadata page 1 (including new entry 593) is to be kept in page 1 444 or moved (depending on the next valid bit of the upper key of the entry). For example, if Figure 5B , for new entry 593, the flash memory system may determine that the next significant bit of the upper key of entry 593 (i.e., bit (N+1) of the upper key of entry 593) is one (1) and store entry 593 in new metadata page 545, while the system may determine that the next significant bit of the upper key of entry 591 is zero (0) and keep entry 591 in metadata page 1. If all KV hash entries of metadata page 1 and the new entry 593 have the same value (e.g., 0) for the next significant bit (e.g., bit (N+1)) (which, while possible, would be unlikely), the flash memory may continue to split the KV hash entries again based on the value of the next significant bit (e.g., bit (N+2)).

[0055] In the case of Figure 5B, after performing the split, the flash memory system can use the fixed-length key 510 to access the corresponding value 505 as follows. The flash memory system can use the upper key with the current UpperKeyIndexLen=N to read the address of the entry 532, and determine two metadata map entries 538, 539 in the overflow area 550 of the metadata map 530 based on the address 542 contained in the entry 532. Next, the flash memory system can determine whether the next significant bit 553 of the upper key 512 (i.e., the (N+1)th bit of the key 510) is zero (0) or one (1). Since the flash memory system determines (554) that the next significant bit 553 is zero, the flash memory system can read the entry 538 (rather than the entry 539). The flash memory system can locate the KV hash entry 591 based on the address 557 contained in the entry 538, and further read the value 505 at the address contained in the entry 591.

[0056] Figure 5C Another example implementation of the flash memory mapping arrangement is shown when the flash memory system splits the KV hash entry of metadata page 1 (544). Figure 5C In some embodiments, when splitting the KV hash entries of metadata page 1 (544), the flash memory system may determine whether each KV hash entry of metadata page 1 (including new entry 593) is to be kept in page 1 444 or moved (depending on the next valid bit of the upper key of the entry). For example, for existing entry 591 of metadata page 1 (544), the flash memory system may determine that the next valid bit of the upper key of entry 591 (i.e., the (N+1)th bit of the upper key of entry 591) is one (1) and move entry 591 from the existing metadata page 544 to the new metadata page 545 (537). In a case having Figure 5C , after performing the split, the flash memory system can use the fixed-length key 510 to access the corresponding value 505 as follows. The flash memory system can use the upper key with the current UpperKeyIndexLen=N to read the address of the entry 532, and determine two metadata map entries 538, 539 in the overflow area 550 of the metadata map 530 based on the address 542 contained in the entry 532. Next, the flash memory system can determine whether the next significant bit 553 of the upper key 512 (i.e., the (N+1)th bit of the key 510) is zero (0) or one (1). Since the flash memory system determines (556) that the next significant bit 553 is one, the flash memory system can read the entry 539 (instead of the entry 538). The flash memory system can locate the KV hash entry 591 based on the address 558 contained in the entry 539, and further read the value 505 at the address contained in the entry 591.

[0057] Figure 6A Illustrated is example pseudo-code for calculating a redirected address using a hash table, according to some embodiments. Figure 6B is a flowchart illustrating an example method for computing redirected addresses using a hash table, according to some embodiments. Figure 6B Flowchart and Figure 6A The pseudo code is consistent.

[0058] refer to Figure 6B In this example, the process begins in step 601 by: a processor (e.g., Figure 2 The processor 250 in FIG. 2 inputs an upper key of a key from a key-value (KV) pair (eg, upper key 412 of key 410 ).

[0059] In step 602, in some embodiments, the processor may utilize the current upper key index length (eg, UpperKeyIndexLen=30) using a mask (eg, Figure 4 420) to calculate the hash table (e.g., Figure 4 Index i of the metadata map 430).

[0060] In step 603, in some embodiments, the processor may retrieve the entry pointed to by index i from the metadata map 430 (eg, Figure 4 432 in the address (e.g., Figure 4 Address 442 in).

[0061] In step 604, in some embodiments, the processor may determine whether the obtained address (eg, address 442) is within the redirected address space (eg, Figure 4 within the redirection address space 464).

[0062] In step 605, in some embodiments, in response to determining that the obtained address is not within the redirected address space, the processor may return the obtained address as a flash address space (eg, Figure 4 The address of the flash address space 462).

[0063] In step 606, in some embodiments, in response to determining that the obtained address (e.g., address 442) is in the redirected address space (e.g., Figure 4 The processor may calculate the address (eg, address 442) from the start address of the redirected address space (eg, Figure 4 The offset from the starting address 468 in (e.g., offset 459).

[0064] In step 607 , in some embodiments, the processor may increase the length of the upper key index (UpperKeyIndexLen) so that the next key position of key 410 may be used.

[0065] In step 608, in some embodiments, the processor may map the metadata 430 to an overflow area (e.g., Figure 4 The starting address of the overflow area 450 in Figure 4 The address or index 451 in the address or index 451) adds an offset (e.g., Figure 4 452 in the offset) and utilize a redirect reference (e.g., Figure 4 The reference 455 in the metadata map 430 updates the index i of the metadata map 430. In some embodiments, the offset 452 may be twice the offset 469 so that the redirected reference 455 may point to two entries 438, 439 of the metadata map 430.

[0066] In step 609, in some embodiments, the processor may use the current UpperKeyIndexLen (which has been incremented in step 607) to update the metadata map 430 by reading one of the two entries pointed to by the updated index i (e.g., Figure 4 ) and obtain the redirected address (e.g., Figure 4 For example, if the next key bit of key 410 is zero (0), the processor may read entry 438 of the two entries 438, 439. Otherwise, if the next key bit of key 410 is one (1), the processor may read entry 439.

[0067] Next, the processor may proceed to step 604 to determine the read entry (eg, Figure 4 In response to determining that the redirected address is still in the redirected address space (e.g., Figure 4 The processor may obtain the new redirected address in the next iteration within the redirected address space 464).

[0068] Figure 7 is a flowchart illustrating an example method for storing key-value (KV) data in non-volatile memory using a hash table according to some embodiments. In this example, the process begins in step 702 by: a processor (e.g., Figure 2 The processor 250 of the controller 220 in the embodiment of the present invention) obtains the key-value (KV) pair (e.g., Figure 3A 301) generates a first key (e.g., Figure 3A 310 in the key).

[0069] In step 704, in some embodiments, the processor may generate a signal based on the first key (e.g., Figure 3A 310 in the volatile memory (eg, Figure 2 DRAM 210) in the hash table (e.g., Figure 3A and 4 The first entry in the metadata map 330, 430) (e.g., Figure 3A In some embodiments, the first entry of the hash table (e.g., Figure 3A The entry 334 in the ) may contain the location of the non-volatile memory (e.g., a 4-byte address of the non-volatile memory indicating Figure 3A The location of metadata page 1(344) in the .

[0070] In step 706, in some embodiments, the processor may generate a query based on the first entry of the hash table (e.g., Figure 3A 334 in the entry) and reads a set of KV hash entries (e.g., Figure 3A The first page of the KV hash entry 391 and the KV hash entry 392 in (eg, Figure 3A 334 in ), each of the set of KV hash entries contains a location in non-volatile memory.

[0071] In step 708, in some embodiments, the processor may determine the first page (eg, Figure 3A Whether the number of entries in the metadata page 334 in the IO has reached a predetermined number (eg, a predefined metadata page size determined based on empirical determination or customer configuration).

[0072] In step 710, in some embodiments, in response to determining that the number of entries in the first page reaches a predetermined number, the processor may store the KV data corresponding to the KV pair (eg, stored in Figure 2 and 4 The value 405 in the flash memory 280 in the non-volatile memory is stored in a first location (e.g., Figure 2 , writes a first KV hash entry containing a first location of non-volatile memory (e.g., containing a value stored in Figure 2 and 4 405 in the flash memory 280 in the KV hash entry 451), and in the second entry of the hash table (eg, Figure 4 438 in the entry ) writes the location of the first KV hash entry (eg, Figure 4 In some embodiments, the second entry of the hash table (e.g., Figure 4The entry 438 in the ) may contain the location of the non-volatile memory (e.g., a 4-byte address of the non-volatile memory indicating Figure 4 The location of the KV hash entry 451 in ).

[0073] In some embodiments, in response to determining that the number of entries in the first page reaches a predetermined number, the processor may generate a new entry based on a second entry of the hash table (eg, Figure 4 The redirect address is calculated based on the index of the entry 438 in Figure 4 The processor may select the redirect address 442 in the third entry of the hash table (e.g., Figure 4 For example, the processor may calculate redirect address 442 by adding start address 468 to offset 469, such that offset 469 is the start address or index of overflow area 450 of entry 438 (e.g., Figure 4 In some embodiments, the redirect address (e.g., Figure 4 The redirection address 442 in the non-volatile memory may be in the address space of the non-volatile memory (e.g., Figure 4 The redirected address space (e.g., Figure 4 within the redirection address space 464).

[0074] In step 712, in some embodiments, in response to determining that the number of entries in the first page has not reached the predetermined number, the processor may store the KV data corresponding to the KV pair in a second location of the non-volatile memory (e.g., the value 305 is stored in Figure 2 and 3A ), and to the first page (eg, Figure 3A 344 in the metadata page 344) adds a new KV hash entry containing a second location in non-volatile memory (e.g., containing Figure 2 and 3A KV hash entry 392 of the location of value 305 in flash memory 280 in ).

[0075] In some embodiments, the hash table may contain a first set of entries in a power-of-two address space (e.g., in Figure 4 ), and a second set of entries outside the power-of-two address space (e.g., in Figure 4 The first set of entries may include the first entry of the hash table (e.g., Figure 3A The second set of entries may include a second entry of the hash table (e.g., Figure 4The number of entries in the second set of entries may be less than the number of entries in the first set of entries (e.g., the number of entries in the overflow region 450 is 10% of the number of entries in the power of two region 440). The method may include a tail distribution based on the number of entries in the page containing the set of KV hash entries (e.g., Figure 1 The number of the second group of entries is determined by the tail part 101 in.

[0076] Figure 8 is a flow chart illustrating an example method for reading KV data in non-volatile memory using a hash table according to some embodiments. In this example, the process begins in step 802 by generating a key (e.g., Figure 4 410 in the key).

[0077] In step 804, in some embodiments, the processor may read an entry of the hash table based on the first key (eg, Figure 4 432 of the metadata map 430 in ).

[0078] In step 806, in some embodiments, the processor may determine the address contained in the entry (e.g., Figure 4 The address 442 contained in the entry 432 in the address space of the non-volatile memory (eg, Figure 4 The redirected address space (e.g., Figure 4 within the redirection address space 464).

[0079] In step 808, in some embodiments, in response to determining the address contained in the entry (e.g., Figure 4 If the address 442 contained in the entry 432 in the hash table is within a redirected address space outside the address space of the non-volatile memory, the processor may calculate an index of the redirected entry of the hash table based on the address contained in the entry (e.g., Figure 4 address or index 455 in .

[0080] In step 810, in some embodiments, the processor may determine the redirected entry of the hash table based on the redirected entry (e.g., Figure 4 438 in the KV hash entry (e.g., Figure 4 KV hash entry 451 at address 457 in ).

[0081] In step 812, in some embodiments, the processor may hash the entries based on the KV (e.g., Figure 4 The KV hash entry 451 at address 457 in the KV pair is read (e.g., stored in Figure 2 and4 405 in the flash memory 280).

[0082] The previous description is provided to enable those skilled in the art to practice the various aspects described herein. Those skilled in the art will readily appreciate various modifications to these aspects, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" (unless specifically stated), but rather "one or more." Unless otherwise specifically stated, the term "some" refers to one or more. All structural and functional equivalents of the elements of the various aspects described throughout the previous description that are known or later known to those skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public, regardless of whether the present disclosure is explicitly stated in the claims. Claim elements are not interpreted as component plus function unless the phrase "component for..." is used to explicitly state the element.

[0083] It should be understood that the specific order or hierarchy of steps in the disclosed processes is an example of an illustrative approach. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the processes can be rearranged while remaining within the scope of the previous description. The accompanying method claims present elements of the various steps in a sample order and are not necessarily intended to be limited to the specific order or hierarchy presented.

[0084] The preceding description of the disclosed embodiments is provided to enable those skilled in the art to make or use the disclosed subject matter. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the preceding description. Therefore, the preceding description is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0085] The various examples illustrated and described are provided merely as examples to illustrate the various features of the claims. However, the features shown and described with respect to any given example are not necessarily limited to the associated example and may be used or combined with other examples shown and described. Furthermore, the claims are not intended to be limited by any one example.

[0086] The foregoing method descriptions and process flow charts are provided only as illustrative examples and are not intended to require or imply that the steps of the various examples must be performed in the order presented. As will be appreciated by those skilled in the art, the order of the steps in the foregoing examples can be performed in any order. Words such as "thereafter," "then," and "next" are not intended to limit the order of the steps; these words are only used to guide the reader in understanding the description of the method. In addition, any reference to a claim element in the singular (for example, using the articles "a," "an," or "the") should not be construed as limiting the element to the singular.

[0087] The various illustrative logical blocks, modules, circuits, and algorithm steps described in conjunction with the examples disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether this functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Although those skilled in the art may implement the described functionality in varying ways for each specific application, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0088] The hardware for implementing the various illustrative logics, logic blocks, modules, and circuits described in conjunction with the examples disclosed herein may be implemented or executed using a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device designed to perform the functions described herein, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some steps or methods may be performed by circuits specific to a given function.

[0089] In one or more exemplary embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. The steps of the methods or algorithms disclosed herein may be embodied in a processor-executable software module that may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium that is accessible to a computer or processor. By way of example and not limitation, such non-transitory computer-readable or processor-readable storage medium may include RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that is accessible to a computer. As used herein, disk and optical disk include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of non-transitory computer-readable and processor-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and / or instructions on a non-transitory processor-readable storage medium and / or computer-readable storage medium that can be incorporated into a computer program product.

[0090] The foregoing description of the disclosed examples is provided to enable one skilled in the art to make or use the present disclosure. Various modifications to these examples will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to some examples without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the examples shown herein, but is to be accorded the widest scope consistent with the appended claims and the principles and novel features disclosed herein.

Claims

1. A device for managing key-value (KV) data in a non-volatile memory, comprising: Controller; and a volatile memory storing a hash table, wherein the controller is configured to: Generate the first key from the key-value KV pair; reading a first entry of the hash table based on the first key, wherein the first entry is located in a non-overflow portion of the hash table; reading a first page including a set of KV hash entries based on the first entry of the hash table, each KV hash entry in the set of KV hash entries containing a length of KV data in the non-volatile memory and an address of the KV data; determining whether the number of KV hash entries of the first page reaches a predetermined number; In response to determining that the number of KV hash entries for the first page reaches the predetermined number, storing KV data corresponding to the KV pair in a first location of the non-volatile memory, writing a first KV hash entry containing the first location of the non-volatile memory as the address in the first KV hash entry, and writing the location of the first KV hash entry in a second entry of the hash table, wherein the second entry is located in an overflow portion of the hash table, the overflow portion having a size smaller than the non-overflow portion of the hash table, and wherein the location of the first KV hash entry written to the second entry of the hash table corresponds to a KV hash entry of a second page separate from the KV hash entries of the first page; and In response to determining that the number of KV hash entries of the first page does not reach the predetermined number, the KV data corresponding to the KV pair is stored in a second location of the non-volatile memory, and a new KV hash entry containing the second location of the non-volatile memory is added to the set of KV hash entries in the first page as the address in the new KV hash entry.

2. The device of claim 1, wherein each of the first page and the second page is located in the non-volatile memory.

3. The apparatus of claim 1 , wherein in response to determining that the number of entries of the first page reaches the predetermined number, the controller is configured to: calculating a redirection address based on an index of the second entry of the hash table; and The redirect address is written into the third entry of the hash table. The apparatus of claim 3 , wherein the redirection address is within a redirection address space outside of an address space of the non-volatile memory.

5. The device of claim 3, wherein the controller is configured to: reading the third entry of the hash table based on the first key; calculating the index of the second entry of the hash table based on the third entry of the hash table; reading the first KV hash entry based on the second entry of the hash table; and The KV data corresponding to the KV pair is read based on the first KV hash entry.

6. The apparatus of claim 5 , wherein, in calculating the index of the second entry of the hash table, the controller is configured to: determining that the address contained in the third entry is within a redirected address space outside the address space of the non-volatile memory; and In response to the determining that the address contained in the third entry is within the redirected address space outside the address space of the non-volatile memory, the index of the second entry of the hash table is calculated based on the address contained in the third entry.

7. The device according to claim 1, wherein the non-overflow portion of the hash table comprises a first set of entries in a power-of-two address space, and the overflow portion of the hash table comprises a second set of entries outside the power-of-two address space, The first set of entries includes the first entry of the hash table, and The second set of entries includes the second entry of the hash table. The apparatus of claim 7 , wherein the number of entries in the second set is less than the number of entries in the first set.

9. The apparatus of claim 7, wherein the controller is configured to determine the number of the second set of entries based on a tail distribution of the number of entries in a page including a set of KV hash entries.

10. The device of claim 1, wherein the controller is configured to: generating the first key from the KV pair; reading the first entry of the hash table based on the first key; reading the first page based on the first entry of the hash table; and The KV data corresponding to the KV pair is read based on the first page.

11. The apparatus of claim 1, wherein in response to determining that the number of entries of the first page reaches the predetermined number, the controller is configured to add a redirection address to the hash table, through which both the first page and the second page are accessible.

12. A method for managing key-value (KV) data in a non-volatile memory, comprising: The processor generates a first key from the key-value KV pair; reading, by the processor, a first entry of a hash table stored in volatile memory based on the first key, wherein the first entry is located in a non-overflow portion of the hash table; Reading, by the processor, a first page comprising a set of KV hash entries based on the first entry of the hash table, each KV hash entry in the set of KV hash entries containing a length of KV data in a non-volatile memory and an address of the KV data; determining, by the processor, whether the number of KV hash entries of the first page reaches a predetermined number; In response to determining that the number of KV hash entries for the first page reaches the predetermined number, storing, by the processor, KV data corresponding to the KV pair in a first location of the non-volatile memory, writing a first KV hash entry containing the first location of the non-volatile memory as the address in the first KV hash entry, and writing the location of the first KV hash entry in a second entry of the hash table, wherein the second entry is located in an overflow portion of the hash table, the overflow portion having a size smaller than the non-overflow portion of the hash table, and wherein the location of the first KV hash entry written to the second entry of the hash table corresponds to a KV hash entry of a second page separate from the KV hash entries of the first page; and In response to determining that the number of KV hash entries of the first page does not reach the predetermined number, the processor stores the KV data corresponding to the KV pair in a second location of the non-volatile memory, and adds a new KV hash entry containing the second location of the non-volatile memory to the set of KV hash entries in the first page as the address in the new KV hash entry.

13. The method of claim 12, wherein each of the first page and the second page is located in the non-volatile memory.

14. The method according to claim 12, further comprising: In response to determining that the number of entries of the first page reaches the predetermined number; calculating a redirection address based on an index of the second entry of the hash table; and The redirect address is written into the third entry of the hash table.

15. The method of claim 14, wherein the redirection address is within a redirection address space outside of an address space of the non-volatile memory.

16. The method of claim 14, further comprising: reading the third entry of the hash table based on the first key; calculating the index of the second entry of the hash table based on the third entry of the hash table; reading the first KV hash entry based on the second entry of the hash table; and The KV data corresponding to the KV pair is read based on the first KV hash entry.

17. The method of claim 16, wherein calculating the index of the second entry of the hash table comprises: determining that the address contained in the third entry is within a redirected address space outside the address space of the non-volatile memory; and In response to the determining that the address contained in the third entry is within the redirected address space outside the address space of the non-volatile memory, the index of the second entry of the hash table is calculated based on the address contained in the third entry.

18. The method according to claim 12, wherein the non-overflow portion of the hash table comprises a first set of entries in a power-of-two address space, and the overflow portion of the hash table comprises a second set of entries outside the power-of-two address space, The first set of entries includes the first entry of the hash table, and The second set of entries includes the second entry of the hash table.

19. The method of claim 18, further comprising: The number of the second set of entries is determined based on a tail distribution of the number of entries in a page containing a set of KV hash entries.

20. The method of claim 12, further comprising: generating the first key from the KV pair; reading the first entry of the hash table based on the first key; reading the first page based on the first entry of the hash table; and The KV data corresponding to the KV pair is read based on the first page.

Images