Key-value storage device and method of sorting keys

Abstract

The present disclosure relates to a key-value storage device. The key-value storage device can include a first storage for persistently storing data. The key-value storage device can also include a second storage for a primary index structure that maps keys to locations in the first storage. A controller can use the first storage to process read requests, write requests, or delete requests from a host. A third storage can store a secondary index structure that stores keys, the secondary index structure being ordered.

Description

Technical Field

[0001] The concepts disclosed herein generally relate to storage devices, and more specifically, to improved ordering of keys in key-value solid-state drives (KV-SSDs). Background Technology

[0002] Key-value solid-state drives (KV-SSDs) store data (the "values" in the key-value pair) at locations on the KV-SSD. These locations are associated with keys (the "keys" in the key-value pair). When a KV-SSD receives a write request, a new key can be added to a structure that maps the key to the location where the data is stored. When a KV-SSD receives a read request, it uses this structure to locate the data storage location so that the data can be read and returned to the requesting machine. And when a KV-SSD receives a delete request, it updates this structure to remove the key from the structure.

[0003] The management of the KV-SSD index structure still needs improvement. Summary of the Invention

[0004] According to one aspect of this disclosure, a key-value (KV) storage device is provided, comprising: a first storage for storing data, the first storage including persistent storage; a second storage for storing a primary index structure that maps a first key to a first location in the first storage for storing values, the first key and the values ​​forming a key-value pair; a controller for using the first storage to process at least one of a read request, a write request, or a delete request received from a host; and a third storage for storing a secondary index structure that stores the first key, wherein the secondary index structure is sorted.

[0005] According to another aspect of this disclosure, a data storage method is provided, comprising: receiving a request from a host at a key-value (KV) storage device, the request including at least an operation and a first key, the operation including at least one of a write operation or a delete operation; processing the request using a first storage device of the KV storage device based at least in part on the operation and the first key, thereby determining a location in the first storage device associated with the first key; updating the main index structure of the KV storage device based at least in part on the operation, the first key, and the location; recording the operation and the first key in a log block of the KV storage device; and returning the result of the request from the KV storage device to the host.

[0006] According to another aspect of this disclosure, a data storage method is provided, comprising: requesting a key from a log block on a key-value (KV) storage device; receiving the key from the KV storage device; and generating a first secondary index structure based at least in part on the key, the first secondary index structure being sorted according to at least the key and a second key, wherein the first secondary index structure is different from a primary index structure, the primary index structure being used by the KV storage device to process read requests including the key, write requests including the key, or delete requests including the key. Attached Figure Description

[0007] The accompanying drawings described below are examples of embodiments of how the inventive concept is implemented, and are not intended to limit the embodiments of the inventive concept. Various embodiments of the inventive concept may include elements not shown in certain drawings and / or elements shown in certain drawings may be omitted. The drawings are intended to provide illustration and may not be drawn to scale.

[0008] Figure 1 A machine including a storage device according to an embodiment of the present disclosure is shown, the storage device being equipped to support cooperative key sorting.

[0009] Figure 2 An embodiment of the invention is shown. Figure 1 Additional details about the machine.

[0010] Figure 3 The operation involved in executing a key record according to an embodiment of the present invention is illustrated.

[0011] Figure 4 The operations involved in executing a merge key according to an embodiment of the present invention are illustrated.

[0012] Figure 5 An embodiment of the invention is shown. Figure 1 Details of the storage device.

[0013] Figure 6 An embodiment of the invention is shown. Figure 5 Key-value command handler Figure 5 Primary key value index structure and Figure 5 The interaction between key loggers.

[0014] Figure 7 An alternative view is shown of the operations involved in executing a merge key according to an embodiment of the present invention.

[0015] Figure 8 The embodiment of the key merging according to the concept of the present invention is shown. Figure 5 In the secondary index structure Figure 1 The processor.

[0016] Figure 9 An embodiment of the invention is shown for use in Figure 1 A flowchart illustrating an example process of executing key records on a storage device.

[0017] Figure 10 An embodiment of the invention is shown for use in Figure 5 The flowchart shows an example process of a key logger using key group metadata to record keys.

[0018] Figure 11A-11B An embodiment of the invention is shown for use in Figure 1 Storage devices to help Figure 1 The processor will merge the keys into Figure 5 The flowchart shows an example process in the secondary index structure.

[0019] Figure 12 An embodiment of the invention is shown. Figure 1 The processor will merge the keys into Figure 5 The flowchart shows an example process in the secondary index structure.

[0020] Figure 13 An embodiment of the invention is shown for use in Figure 1 The processor from Figure 3 A flowchart of an example procedure for accessing log block keys.

[0021] Figure 14 An embodiment of the invention is shown. Figure 1 Key-value storage devices will merge keys into Figure 5 The flowchart shows an example process in the secondary index structure.

[0022] Figure 15 An embodiment of the invention is shown. Figure 1 The processor will key and Figure 8 A flowchart illustrating an example process of merging the second secondary index structure.

[0023] Figure 16 An embodiment of the invention is shown. Figure 1 The processor requested deletion Figure 3 A flowchart of an example process for logging blocks. Detailed Implementation

[0024] Reference will now be made in detail to embodiments of the inventive concept, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description to enable a thorough understanding of the inventive concept. However, it should be understood that those skilled in the art can practice the inventive concept without these specific details. In other instances, well-known methods, processes, components, circuits, and networks have not been described in detail to avoid unnecessarily obscuring aspects of the embodiments.

[0025] It should be understood that although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the inventive concept, a first module may be referred to as a second module, and similarly, a second module may be referred to as a first module.

[0026] The terminology used herein to describe the inventive concept is for the purpose of describing particular embodiments only and is not intended to limit the inventive concept. As used in the description of the inventive concept and the appended claims, the singular forms “a,” “an,” and “this” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It will be further understood that when the terms “comprising” and / or “including” are used in this specification, the presence of the stated feature, integer, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integers, steps, operations, elements, and / or components, and / or groups thereof, is not excluded. Components and features in the accompanying drawings are not necessarily drawn to scale.

[0027] Key-value storage devices can use a primary index structure to handle various requests, such as write, read, or delete requests. Because these requests represent the majority of requests that might be received from the requester, updates to this structure can interfere with other operations, particularly input / output (I / O) operations. In other words, while the primary index structure is being updated, the key-value storage device may not be able to handle write, read, or delete requests for other data stored in the key-value storage device. Furthermore, since this structure can be stored in flash memory like user data, updating this structure may involve invalidating the existing structure and writing the updated structure to a new location in flash memory. This process can lead to an increase in the write amplification factor (WAF). On the other hand, because KV-SSDs use this structure, updates to this structure may not be delayed indefinitely.

[0028] In various aspects, range queries can be a characteristic of key-value stores. In some key-value stores, responding to range queries involves two steps: logging and merging (compression). During the logging phase, the key-value store writes unsorted or partially sorted keys to a solid-state drive (SSD), and during the merging phase, the list of logged keys is read and combined with the main index tree. Note that key-value stores can perform these operations even if no range query is pending.

[0029] However, supporting this processing internally in key-value SSDs (KV-SSDs) can be challenging due to limitations in internal processing power. Records can be implemented internally, but merging and sorting can be computationally expensive processes for the device. Furthermore, while general-purpose processors (or hardware specifically designed for merging, such as appropriately configured field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs)) can be used, dedicating hardware to these specific tasks can increase the overall cost and / or complexity of key-value storage.

[0030] In some respects, KV-SSDs can provide relatively fast and scalable key-value pair management. However, sorting incoming key-value pairs to support range queries can lead to performance degradation. Updating the primary index structure on demand is expensive and doesn't scale well with the number of keys. In some cases, a two-step sorting technique can be used, which partially processes incoming key-value pairs and stores them on the storage device while organizing the stored data in the background. However, this approach can have some performance issues. Records need to be written synchronously, which KV-SSDs may not provide. Furthermore, merging updates to the primary index structure can interfere with the processing of incoming input / output (I / O) requests.

[0031] To address these performance issues, atomic I / O operations can be used to offload records to the KV-SSD: the host can avoid executing records or issuing record requests to the KV-SSD. Merging can be performed by the host, and the processed results are stored separately in the KV-SSD (or elsewhere) in another structure without interfering with the main index structure.

[0032] KV-SSDs may include a key logger, which writes updated or deleted keys to persistent memory or storage media within the device. KV-SSDs may also support two commands: an iterative command to retrieve write buffers written to or maintained by the key logger, and a log deletion command to delete write buffers written to or maintained by the key logger. Furthermore, the write and delete commands can be modified to support new metadata fields for key grouping. This key group metadata can be used for recording and sorting: keys assigned to the same key group can be sorted together. If no key group metadata is specified, the KV-SSD determines that all keys are in the same group.

[0033] Key group information can be passed via write / delete key-value I / O commands, which can be implemented over native I / O command protocols such as Non-Volatile Memory Express (NVMe). Key group information can be passed from optional fields in a specific I / O command protocol or encoded in the key itself. The key-value command handler can pass key group metadata from write and delete commands to the key logger.

[0034] A key logger can store keys into each set of write buffers residing in persistent memory. This persistent memory can include non-volatile memory, byte-addressable dynamic random access memory (DRAM) (which can be battery-powered), byte-addressable non-volatile memory, and other forms. Each key log entry can contain the key (its length and key content) and information about the key, such as the OPCODE (indicating write / delete) and optional metadata (such as group information).

[0035] If the persistent memory storing the write buffer lacks sufficient remaining space, the write buffer can be moved to a persistent storage medium such as flash memory. The location of the write buffer in flash memory can be maintained by the device. The write buffer can be allocated in a series of logical block addresses (LBAs) in memory, stored using key-value pairs for write buffers in persistent storage, or otherwise stored. For example, a region in flash memory can be reserved for write buffers. The key logger can maintain a list of free LBAs in that region and allocate write buffers using the free LBAs. The region can be reclaimed when a log deletion command is issued.

[0036] Another method for managing write buffers is to assign unique keys to the write buffers and store them as key-value pairs in the main memory of the flash memory. A key logger can maintain a list of assigned unique keys and return them during iteration commands. When the host issues an iteration command, the key logger can return these keys for the write buffers, and the host can read the write buffers using a standard read request. Once the write buffers have been read from the main memory in flash memory, the host can issue a request to delete these keys. Using this method, KV-SSDs can eliminate the need for log deletion commands (because the host can directly delete the write buffers using a delete request and the assigned keys).

[0037] Iteration commands can retrieve write buffers held or written by the key logger. Iteration commands can be implemented in two ways: either they can return the contents of the write buffer, or they can return a list of unique identifiers (e.g., keys and LBAs) for the write buffer.

[0038] If the amount of data to be returned is too large for a single command, multiple iteration commands may be issued. Table 1 shows an example implementation of an iteration command using a pre-allocated LBA range, along with sample code of the host using this command:

[0039] Table 1

[0040]

[0041] If you access the contents of the write buffer using an iteration command, you can issue a log deletion command to erase the write buffer when the iteration command completes. If the iteration command returns an identifier of the write buffer (e.g., a key or LBA), the log deletion command can be omitted, because the host can erase the write buffer using a deletion request and the key assigned to the write buffer.

[0042] The merge process performs the following three tasks: reading the write buffer; generating a sorted list of keys (and writing them to the KV-SSD); and deleting from the write buffer. As mentioned above, the write buffer can be read using an iterative command.

[0043] To generate a sorted list of keys, the host can retrieve key entries from the write buffer. The host can also load a memory index node containing the key entries (read from the write buffer). The host can then insert new keys into the memory index node and remove deleted keys from it. The host can then write the updated metadata to the KV-SSD (but using a different structure than the primary index structure).

[0044] Finally, to delete the write buffer, the host can issue a log delete command to delete the write buffer (or, if the write buffer is stored as a key-value pair on the KV-SSD, the host can use a delete request with the key assigned to the write buffer).

[0045] To perform the merge itself, any indexed data structure with the required sorting can be used, including, for example, B+-tree, B-tree, skip list, and others.

[0046] Because the sorted index structure can use the write buffer (and / or the main index structure) in the event of a power failure or write failure (the processed write buffer may be deleted when all metadata nodes have been successfully written to the KV-SSD), the sorted index structure is consistent.

[0047] In summary, when the primary index structure is used to store information about keys in a sorted manner, it can be updated multiple times in a way that affects the entire primary index structure. While the primary index structure is being updated, other I / O operations may be interrupted or delayed (because I / O operations have access to the primary index structure). Furthermore, KV-SSDs can perform multiple write operations to ensure data consistency. New keys can be written to both the log and temporary files (e.g., as part of a write buffer).

[0048] By using a secondary index structure instead of a primary index structure to store sorted key information, KV-SSD avoids the need to update the primary index structure, during which I / O operations may be interrupted or delayed. Additionally, since less data in the primary index structure can be modified, the WAF (Web Application Firewall) of data stored on KV-SSD can be reduced. Finally, fewer temporary copies of the keys can be written to the storage device.

[0049] Figure 1 A machine including a storage device, configured to support cooperative key ordering, is illustrated according to an embodiment of the present invention. Figure 1 The diagram shows a machine 105, which may also be referred to as a host. Machine 105 may include a processor 110. Processor 110 may be any type of processor. (For ease of illustration, processor 110 and other components discussed below are shown externally to machine 105; embodiments of the inventive concept may include these components internal to machine 105.) Although Figure 1A single processor 110 in machine 105 is shown, but machine 105 may include any number of processors, each of which may be a single-core or multi-core processor, each of which may implement a Reduced Instruction Set Computer (RISC) architecture or a Complex Instruction Set Computer (CISC) architecture (among other possibilities), and may be mixed in any desired combination.

[0050] Machine 105 may also include memory 115. Memory 115 may be any type of memory, such as flash memory, dynamic random access memory (DRAM), static random access memory (SRAM), persistent random access memory, ferroelectric random access memory (FRAM), or non-volatile random access memory (NVRAM), such as magnetoresistive random access memory (MRAM). Memory 115 may also be any desired combination of different memory types. Machine 105 may also include a memory controller 120, which can be used to manage access to memory 115.

[0051] Machine 105 may also include storage device 125. Storage device 125 can be used to store data. Processor 110 may run device driver 130, which can support access to storage device 125. While embodiments of the inventive concept may focus on key-value storage devices, any desired storage device can be used, which can operate using any desired storage principle. Thus, storage device 125 may be a solid-state drive (SSD), hard disk drive, or any other desired storage device. Although Figure 1 Only one storage device 125 is shown, but embodiments of the inventive concept can support any number of storage devices installed in machine 105, which may differ from each other (or be similar or identical to each other) in any particular individual element.

[0052] Key-value storage devices, such as storage device 125, use keys to identify data instead of logical block addresses (LBAs). Unlike block-based storage (where data is expected to be written and read in units of specific, predefined sizes, such as pages or blocks), objects can be conceived as arbitrary sizes. The size of an object stored on a key-value storage device can be limited by the capacity of the key-value storage device. Therefore, the size of an object can be smaller than a page or block, or the size of an object can be larger than a page or block (although the size of a page or block still controls how the object is stored, how the storage manages it is separate from how the object is written or read).

[0053] Similarly, while block-based storage can expect LBAs to fit a specific range of values ​​(and therefore use specific predefined bits in the LBA), keys can be conceived as arbitrary sizes and can have any desired value. Because the number of bits in the key can vary, key-value stores can be more flexible than block-based storage. However, there are other considerations. For example, while different applications might use unique LBAs, nothing prevents different applications from attempting to write data using the same key. In this case, the key-value store can return an error to the second application, informing it that the key value is already in use and therefore cannot be written to the key-value store.

[0054] One type of request that can be received by a key-value store (such as key-value storage device 125) is called a range query. A range query can request information about keys that satisfy specific conditions. For example, a range query might request all keys associated with values ​​within a specific range, or all keys existing in the key-value store where the keys themselves fall within a specific range. Sorting the keys can speed up the processing of such queries.

[0055] Key-value storage device 125 can use a primary index structure to store and manage key-value pairs. In some key-value SSDs, the primary index structure can also store such information about the keys in a sorted manner. However, such requests can affect the sorting order of keys each time a new key is written or an old key is deleted. Therefore, when writing or deleting a key, it may be necessary to update the entire primary index structure (and since key-value SSDs typically use flash memory, in-situ modification is not possible, so the old primary index structure is deleted and a new primary index structure is written). It is desirable that the information stored in the primary index structure is consistent: that is, the data actually written to the key-value storage device should be reflected by the primary index structure. Therefore, when the primary index structure is updated, the key-value storage device is typically unable to handle input / output (I / O) requests. Thus, some key-value storage devices may be "blocked" from processing I / O requests when the primary index structure is updated, thereby increasing the time required for the key-value SSD to respond to I / O requests.

[0056] In addition to the performance impact of updating the master index structure, since flash memory does not support in-situ data modification, updating the master index structure also increases the number of program-erase cycles a block undergoes. Because cells in flash memory have a limited number of program-erase cycles before their reliability may be compromised, updating the master index structure can lead to premature wear and tear on the key-value storage device. This result is known as the write amplification factor (WAF) of data written to key-value storage device 125.

[0057] One way to address these issues is to delay or avoid updating the primary index structure. However, since the primary index structure is used to handle I / O requests, updating the primary index structure may not be delayed indefinitely.

[0058] This solution also assumes that a write-ahead log can be used to ensure data consistency within the primary index structure. However, if the key-value storage device 125 supports atomic write operations, the use of a write-ahead log can be omitted. Atomic operations can be used to perform updates to the primary index structure to include new key-value pairs (albeit not in sorted order), which ensures either a complete update of the primary index structure or no update at all.

[0059] In embodiments of the inventive concept, a secondary index structure may be introduced. This secondary index structure may (to some extent) reflect the information in the primary index structure, but the keys stored for management by the secondary index structure may be ordered. In some embodiments of the inventive concept, the secondary index structure may include keys from key-value pairs, but not the (physical) location on the key-value storage device 125 where the data is stored. This fact may be a result of key recording being performed simultaneously with (or before) writing the key-value pairs to the actual memory in the key-value storage device 125: if the location is not yet known, it may not be recorded, and therefore may not be included in the secondary index structure. In other embodiments of the inventive concept, the location of the stored value on the key-value storage device 125 may be included in the secondary index structure by having the location added to the information recorded during key recording, or by accessing the location from the primary index structure or the flash memory translation layer.

[0060] Using secondary index structures helps solve the problems associated with using primary index structures to store sorting information about keys. Primary index structures are updated very quickly: new keys may be written to the end of the primary index structure, and deleted keys may be marked as invalid. Furthermore, secondary index structures can be updated without interfering with the use of the primary index structure, thus avoiding any performance issues caused by delayed I / O requests.

[0061] While using a secondary index structure may require additional space (depending on the information stored in the secondary index structure, the size of the secondary index structure may be comparable to that of the primary index structure, meaning that the storage required to store both index structures is approximately twice that required to store only the primary index structure), as mentioned above, there are advantages to using a secondary index structure: the primary index structure can still be used even if the secondary index structure is being updated.

[0062] Additionally, machine 105 can store the secondary index structure in memory 115 and can use the secondary index structure to handle range queries. Therefore, the secondary index structure does not need to be stored on key-value storage device 125. Storing the secondary index structure on key-value storage device 125 avoids the need to recreate it later (which would involve reading the primary index structure and sorting that information, potentially a time-consuming process if the primary index structure contains thousands or millions of keys). However, when stored on key-value storage device 125, the secondary index structure may not be updated as frequently as the primary index structure, thus reducing WAF (Web Application Firewall).

[0063] Because the secondary index structure is updated using a merge sorting process based on information stored in log blocks about updates to key-value storage device 125, the information in the secondary index structure can differ from the information in the primary index structure. That is, the primary index structure can include keys that have not yet been added to the secondary index structure, and keys that have not yet been deleted from the secondary index structure can be omitted. In short, the number of keys in the secondary index structure (which can be zero or more during the lifetime of key-value storage device 125) can differ from the number of keys in the primary index structure.

[0064] Secondary index structures can use any desired data structure. Example data structures that can be used include B+ trees, B-trees, time-slot pages, or skip lists. Additionally, information about the keys (and other information) stored in the data structure can be stored in time-slot pages (time-slot pages can be used to efficiently store keys of different sizes).

[0065] although Figure 1 Machine 105 is described as a server (which may be a standalone or rack-mounted server), but embodiments of the inventive concept may include any desired type of machine 105 and are not limited thereto. For example, machine 105 may be replaced by a desktop or laptop computer or any other machine that may benefit from embodiments of the inventive concept. Machine 105 may also include dedicated portable computing machines, tablet computers, smartphones, and other computing machines. Additionally, applications that can access data from storage device 125 may reside in a separate machine from machine 105 and access machine 105 via a network connection traversing one or more networks of any type (wired, wireless, global, etc.).

[0066] Figure 2 An embodiment of the invention is shown. Figure 1 Additional details about Machine 105. Figure 2 Typically, machine 105 includes one or more processors 110, which may include a memory controller 120 and a clock 205 for coordinating the operation of components of device 105. Processor 110 may also be coupled to memory 115, which, for example, may include random access memory (RAM), read-only memory (ROM), or other state-preserving media. Processor 110 may also be coupled to storage device 125 and network connector 210, which may be, for example, an Ethernet connector or a wireless connector. Processor 110 may also be connected to bus 215, which may connect user interface 220 and input / output interface ports, which can be managed using input / output engine 225 and other components.

[0067] Figure 3 The operations involved in executing a key record according to an embodiment of the present invention are illustrated. Figure 3 In this context, host 105 can issue key-value request 305. (In practice, key-value request 305 can be issued from an application running on host 105: for the purposes of this discussion, the distinction between host 105 and the application running on host 105 can be ignored.) As a result of key-value request 305, host 105 can issue write request 310, which can be sent to key-value storage device 125. Key-value storage device 125 can then perform two operations: the key-value pair that is the object of write request 310 can be written to the main index structure (operation 315), and the operation can be written to a log block (operation 320), such as log block 325 (which can also be referred to as a logbook and can be used similarly to write log or write buffer). Operation 315 can update the main index structure, which can be used by key-value storage device 125 to store keys (and associate the location of the value in memory with the key so that the value can be retrieved or deleted later).

[0068] Operation 320 may store information about write request 310 in log block 325. The stored information may include operation (or opcode) 330-1 (which may indicate whether the request is to write a new key-value pair to key-value storage device 125 or to update the value associated with an existing key to the new value). The stored information may also include key 335-1, indicating the key affected by the request.

[0069] Log block 325 may also store additional information, marked as option 340-1 (which may also be referred to as metadata). This additional information can be considered metadata of the request. Examples of such metadata may include metadata related to the size of the value, key, or value in a key-value pair, as well as other metadata. Option 340-1 may be included as an option or metadata as part of write request 310, or option 340-1 may be derived from write request 310.

[0070] Another example of metadata that can be used is key group metadata. Key group metadata can be used to associate a specific request with a key group. By grouping keys together, keys can be organized in different arrangements.

[0071] As an example of how key groups can be used, consider a scenario where a user stores three different types of data on key-value storage device 125: ordinary data and two types of metadata. By assigning each type of data to a different key group, requests involving each type of data can be grouped together. This arrangement allows keys (when sorted into a secondary index structure, such as reference) Figure 4 , 7 (As further described in sections -8, 12-14, and 15) are divided into different structures for different key groups. For example, log block 325 is shown as associated with key group 1: requests associated with other key groups can be logged in other log blocks associated with those other key groups. Therefore, although Figure 3 Only one log block 325 is shown, but embodiments of the present invention may include any number (zero or more) of log blocks (although zero log blocks may indicate that the secondary index structure is up-to-date, this may not happen frequently).

[0072] Host 105 can also issue requests that do not require logging. For example, key-value storage device 125 can be used to store temporary files, journaling information, or metadata. Objects representing this data may not need to be sorted into the secondary index structure. Key group metadata can be used to identify keys that should not be logged so that they can be merged into the secondary index structure later.

[0073] Log block 325 is shown as containing multiple tuples. Log block 325 can include multiple tuples because multiple requests may be logged between updates to the secondary index structure. Therefore, opcode 330-1, key 335-1, and option 340-1 can represent information associated with the first request, opcode 330-2, key 335-2, and option 340-2 can represent information associated with the second request, and opcode 330-3, key 335-3, and option 340-3 can represent information associated with the third request. Although Figure 3The information shown relates only to the three requests in log block 325, but embodiments of the present invention may store information related to any number (zero or more) of requests in log block 325.

[0074] at last, Figure 3 This is described with reference to write request 310. However, embodiments of the inventive concept can be extended to record information about other requests. For example, in the simplest implementation, the key-value store can provide three basic commands: GET (get) (read data from the key-value store), PUT (put) (write data to the key-value store), and DELETE (remove data from the key-value store). Although the above description uses GET, PUT, and DELETE to identify these commands, embodiments of the inventive concept can include key-value stores that use different names (opcodes) for these commands. The GET command typically does not modify data on the key-value store, but the PUT and DELETE commands can modify data on the key-value store. Therefore, the DELETE command can cause... Figure 3 The operation shown is similar to the PUT command.

[0075] Furthermore, although GET, PUT, and DELETE represent central commands that can be processed by a key-value store, embodiments of this invention can be extended to key-value stores that provide additional commands. Any such command that may affect information stored in a secondary index can lead to... Figure 3 The operation shown is similar to the operation shown.

[0076] Figure 4 The operations involved in performing a key merging according to an embodiment of the present invention are illustrated. To merge new key information into the secondary index structure, host 105 may request a list of keys that have been modified (i.e., written or deleted) on key-value storage device 125 (as shown in operation 405). Key-value storage device 125 can then provide a list of keys that have been modified since the last update of the secondary index structure (shown as operation 410). For example, this list of keys may include those from… Figure 3 The information in log block 325. Once host 105 receives the list of keys that have been updated since the last update of the secondary index structure, host 105 can perform a merge sort (as shown in operation 415). This operation may involve updating... Figure 1 The secondary index structure in memory 115.

[0077] Then, host 105 can update the secondary index structure stored on key-value storage device 125 (operation 420). When host 105 sends a request to key-value storage device 125 to update the secondary index structure, key-value storage device 125 can then write the sorted keys into the secondary index structure (as shown in operation 425). Finally, host 105 can send a request to clear key-value storage device 125 (as shown in operation 430). This operation may involve deleting key-value storage device 125. Figure 3 Log block 325 (or at least from) Figure 3 Log block 325 deletes data, potentially leaving... Figure 3 Log block 325 is used to record future updates to key-value information stored on key-value storage device 125.

[0078] Figure 5 An embodiment of the invention is shown. Figure 1 Details of storage device 125. Figure 5 In this configuration, the KV-SSD 125 may include a host interface 505 (which may also be referred to as host interface logic or HIL), an SSD controller 510, and various flash memory chips 515-1 to 515-8 (also referred to as flash memory modules), which may be organized into various channels 520-1 to 520-4. The host interface logic 505 can manage the KV-SSD 125 and other components (e.g., Figure 1 Communication between the processor 110 and the processor 125. In some embodiments of the present invention, the KV-SSD 125 may also support its own network connectivity (and communication with the processor 110 via the processor 110). Figure 1 (The network interface of machine 105 is relative to): In these embodiments of the present invention, the host interface logic 505 can also manage communication with devices remote from the KV-SSD 125: that is, not considered as part of the network interface of machine 105. Figure 1 The device is part of machine 105, but communicates with KV-SSD 125. These communications may include read requests to read data from KV-SSD 125, write requests to write data to KV-SSD 125, and requests to delete data from KV-SSD 125. Host interface logic 505 may manage an interface across a single port, or it may manage an interface across multiple ports. Alternatively, KV-SSD 125 may include multiple ports, each with a separate host interface logic 505 to manage the interface across that port. Embodiments of the inventive concept may also combine various possibilities (e.g., an SSD with three ports may have one host interface logic to manage one port and a second host interface logic to manage the other two ports). To support direct communication with remote devices over a network, HIL 505 may include an Ethernet component ( Figure 5 (not shown in the image) or some equivalent network communication components.

[0079] The SSD controller 510 can use a flash memory controller ( Figure 5 (Not shown) Manages read and write operations, garbage collection, and other operations on flash memory chips 515-1 to 515-8. The SSD controller 510 may include a flash translation layer 525, a primary index structure 530, a secondary index structure 535, a key-value command handler 540, and a key logger 545. The SSD controller 510 can be implemented using some form of hardware: for example, as a custom controller for the KV-SSD 125, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) with appropriate software and configuration, or some type of general-purpose processor (such as a central processing unit (CPU), graphics processing unit (GPU), or general-purpose GPU (GPGPU)).

[0080] The flash memory translation layer 525, also known as the object translation layer, can manage the flash memory translation layer. Figure 1 The machine 105 uses a conversion between logical data identifiers (i.e., the keys of key-value pairs) and physical locations in flash memory chips 515-1 to 515-8 (where the values ​​identified by the logical data identifiers are actually stored). In some embodiments of the present invention, the flash memory translation layer 525 may include a primary index structure 530 (as shown by the line connecting the flash memory translation layer 525 and the primary index structure 530) and / or a secondary index structure 535; in other embodiments of the present invention, the primary index structure 530 and / or the secondary index structure 535 may be separate from the flash memory translation layer 525.

[0081] The primary index structure 530 and the secondary index structure 535 have already been discussed above and will not be repeated here. The key-value command handler 540 can... Figure 1 The host 105 (or another machine located away from the KV-SSD 125) receives and processes requests (also referred to as commands). In addition to reading, writing, and deleting data from flash memory chips 515-1 to 515-8, the key-value command handler 540 can be responsible for (or assist in) updating the master index structure 530 and passing information to the key recorder 545. The key-value command handler 540 may also be adapted to process additional commands specifically for cooperative key sorting, which may include commands from... Figure 3 Log block 325 reads information about the request and / or from Figure 3 Log block 325 deletes such requests. Key logger 545 can store information about the request in... Figure 3The key logger 545 may also include additional information not included in the request, such as the (physical) location of the stored value in flash memory chips 515-1 to 515-8. In some embodiments of the inventive concept, the key logger 545 may wait to store the information in log block 325. Figure 3 In log block 325, this continues until all the information to be logged is available. See below for reference. Figure 6 Further discussion of key-value command handler 540 and key logger 545.

[0082] Although Figure 5 The SSD controller 510 is shown to include a flash translation layer 525, a primary index structure 530, a secondary index structure 535, a key-value command processor 540, and a key logger 545; however, embodiments of the inventive concept can position these modules wherever desired. Embodiments of the inventive concept can also position these modules within different portions of the KV-SSD 125: for example, none of these modules may be located within the SSD controller 510. Specifically, the KV-SSD 125 may include additional storage elements 550-1 and 550-2, which can be used to store the primary index structure 530, the secondary index structure 535, and other information, such as... Figure 3 Log block 325. Additional storage elements 550-1 and 550-2 may be "reserved": that is, memory that cannot be used for user data, such as flash memory chips 515-1 to 515-8. Alternatively, some or all of these components (e.g., flash translation layer 525, primary index structure 530, and / or secondary index structure (535)) may be stored in flash memory chips 515-1 to 515-8.

[0083] Additional storage elements 550-1 and 550-2 can be implemented using any desired hardware. For example, additional storage elements 550-1 and 550-2 can be implemented using non-volatile memory (such as NAND flash memory, such as flash chips 515-1 to 515-8), battery-supported volatile memory (such as some form of battery-supported RAM), or volatile memory without a backup battery (although in the latter case, the information stored in the additional storage element may be data that does not need to be persisted, or can be recreated from other persistent data in the event of data loss due to a power outage). Additional storage elements 550-1 and 550-2 can use block-addressable memory, byte-addressable memory, or key-value memory to store data. Furthermore, additional storage elements 550-1 and 550-2 can be of different kinds, both in terms of the hardware used to store data and the manner in which that data can be addressed. Finally, although... Figure 5Only two additional storage elements, 550-1 and 550-2, are shown, but any number (zero or more) of such additional storage elements may exist.

[0084] Key-value storage device 125 may also include Figure 5 Additional memory, not shown, such as additional volatile memory (e.g., various types of RAM) or additional non-volatile memory. The main index structure 530 can be loaded into this additional memory, and updates to the main index structure 530 can be performed in this additional memory. The main index structure 530 can then be flushed back to main memory (e.g., flash memory chips 515-1 to 515-8), depending on the situation. For example, after each change to the main index structure 530, the main index structure 530 can be flushed back to main memory. Alternatively, when Figure 1 When host 105 performs a merge operation on secondary index structure 535, primary index structure 530 can be written to primary storage (in case the update of primary index structure 530 is lost for some reason). Figure 3 Log block 325 may also serve as an update log for the primary index structure. If the primary index structure 530 is too large to be loaded into the additional storage, the key-value storage device 125 may also partially load the primary index structure 530 into the additional storage, and updates will be written back to the key-value storage device 125 as appropriate or when different parts of the primary index structure 530 are to be loaded into the additional storage.

[0085] Although Figure 5 The KV-SSD 125 is shown as comprising eight flash memory chips 515-1 to 515-8 organized into four channels 520-1 to 520-4; however, embodiments of the present invention can support any number of flash memory chips organized into any number of channels. Similarly, although Figure 5 The structure of an SSD is shown, but other storage devices (such as hard drives) can be implemented using different structures, but with similar potential benefits.

[0086] Figure 6 An embodiment of the invention is shown. Figure 5 Key-value command handler 540 Figure 5 Primary key value index structure and Figure 5 The interaction between key loggers 545. Figure 6In this embodiment, key-value command handler 540 is shown to be capable of receiving multiple different requests. These requests may include requests associated with accessing user data from memory in storage device 125, such as write request 605, delete request 610, and other requests 615 (which may include, for example, read requests). Write request 605, delete request 610, and other requests 615 may include support for metadata used in the key record. Note that not all requests associated with user data necessarily affect the key record: for example, writing a new key-value pair and deleting an old key-value pair may affect... Figure 5 The secondary index structure 535 may not involve updating or reading values ​​that pair with keys on or from storage device 125 without involving key records. However, within the scope of any requests supported by key-value command handler 540 that may involve key logs, these requests may include metadata used in key records.

[0087] When the key-value command handler 540 receives a request involving a key record, it can pass the request (or at least relevant information such as the operation, key, and options) to the key logger 545. The key logger 545 can then write this information to log block 325. The key logger 545 can update log block 325 with new information in any desired manner. The key logger 545 can also obtain additional information from other sources that may be included in log block 325, such as the physical location where the value paired with the key is stored. By using append operations (i.e., writing new information to the end of log block 325), log block 325 can also maintain the order in which the key logger 545 processed requests, which can be useful information. The key-value command handler 540 (and the generally used storage device 125) can also process requests normally in other ways, update the main index structure 530 appropriately, and manage the storage in other ways (via I / O engine 620). Figure 5 Data in flash memory chips 515-1 to 515-8.

[0088] In some cases, the key logger 545 may need to do more than just write information to log block 325. For example, consider the case where log block 325 is full (i.e., there is no additional space to store updates in log block 325). In this case, the key logger 545 can store log block 325 somewhere and begin storing updates in a new log block. The key logger 545 can store log block 325 as key-value pairs. Figure 5 The flash memory chips 515-1 to 515-8 (same as the user data that can be stored). Alternatively, the key logger 545 can store the log block 325 in a dedicated memory, such as in... Figure 5The additional storage elements 550-1 and / or 550-2 are also included. The key logger 545 can also store the log block 325 in multiple locations: for example, in local memory of the key logger 545 (such as RAM or battery-supported RAM), and in persistent memory (such as...). Figure 5 Additional storage elements 550-1 and / or 550-2).

[0089] To update the primary index structure 530, the key-value command handler 540 can simply append any relevant information to the primary index structure 530. Recall, Figure 5 The secondary index structure 535 can store information from the primary index structure 530 in a sorted manner to aid range queries; therefore, the primary index structure 530 can store information in an unsorted manner. In this case, a simple append operation is sufficient to update the primary index structure 530.

[0090] The key-value command handler 540 may also include support for additional requests related to information in the management log block 325. These requests may include iteration requests 625 and log deletion requests 630. Iteration requests 625 can be used to access keys and other information from log blocks 325 that have not yet been merged into the management log block. Figure 5 The secondary index structure 535. When the key-value command handler 540 receives an iteration request 625, the key-value command handler 540 can pass the iteration request 625 to the key logger 545. Then, the key logger 545 can read information about the request from the log block 325, such as... Figure 3 Operations 330-1, 330-2, and 330-3, Figure 3 The keys 335-1, 335-2, and 335-3, and Figure 3 Options 340-1, 340-2, and 340-3, and return this information to Figure 1 Host 105.

[0091] In embodiments of the inventive concept where key groups (or other metadata) can be used, iteration request 625 may include an identifier of the key group (or other metadata) of interest. In this way, key-value storage device 125 can return information of interest. Figure 1 Host 105.

[0092] Iteration request 625 can also include other parameters. For example, Figure 1 The host 105 can specify a number as a parameter, which can represent the key-value storage device 125 returns to Figure 1 The maximum number of updates for host 105. For example, Figure 1Host 105 can request, for example, 20 updates in response to iteration request 625. If log block 325 contains more than twenty entries, information about the first 20 entries can be returned, and the remaining entries can remain in log block 325 for later iterations. Therefore, Figure 1 Host 105 can send multiple iteration requests 625 to retrieve information about all updates in log block 325.

[0093] There may also be limitations on how much information key-value storage device 125 can return in response to iteration request 625. For example, Figure 1 The host 105 may have allocated a fixed amount of space, for example, approximately 4KB, for information about updates to key-value pairs in the key-value storage device 125. This fixed amount of space may, for example, correspond to the size of log block 325. If log block 325 contains more data than the amount of data that can be returned in the specified space, or if there are enough updates to fill more than one log block 325, the key logger 545 may return only a subset of the updates recorded by the key logger 545. The key logger 545 may instruct a return to Figure 1 The key logger 545 may also use other techniques to indicate the presence of additional updates awaiting retrieval, such as: Figure 1 The host 105 can simply send an iteration request 625 until the key logger 545 responds indicating that no further updates are pending. Therefore, in some embodiments of the inventive concept, Figure 1 Host 105 can issue multiple iterative requests 625 to retrieve all updates.

[0094] The key logger 545 can respond to the iteration request 625 in many different ways. For example, the key logger 545 can simply read information about the update from the log block 325 and send that information back to the host 105. If the log block 325 is stored in any memory, such as... Figure 5 Additional storage elements 550-1 and / or 550-2, or in the local memory (such as RAM) of the key recorder 545, Figure 1 Host 105 may not be able to directly access information from log block 325 and may require key logger 545 to provide updates from log block 325 in response to iteration request 625 (if log block 325 is stored in persistent storage, such as...). Figure 5If additional storage elements 550-1 and / or 550-2 are provided, the key logger 545 may load log block 325 into its local memory to expedite access to information in log block 325. Alternatively, if log block 325 is written like any other key-value pair... Figure 8 In the flash memory chips 515-1 to 515-8, the key logger 545 can return only the keys associated with the log block 325. Figure 1 Host 105. Then, host 105 can issue a read request to read the information in log block 325.

[0095] Upon receiving log deletion request 630, key logger 545 can then delete information from log block 325. The information deleted in response to log deletion request 630 may be information about previous iterations as a result of iteration request 625. Figure 1 The information requested by host 105.

[0096] The key logger 545 can handle log deletion requests 630 in many ways. For example, the key logger 545 can store information about logs sent to... Figure 1 The key logger 545 can then delete all information in log block 325 up to the tracked key upon receiving log deletion request 630, or it can directly delete log block 325. (Recall that one of the advantages of embodiments of the inventive concept is that other I / O operations can continue uninterrupted due to the merge sort operation). Even during or after iteration request 625 is processed, key logger 545 can add new information to log block 325. Therefore, key logger 545 may not assume that all information in log block 325 has been sent to host 105 in response to iteration request 625.

[0097] It may also happen Figure 1 Host 105 can request some (not all) information from log block 325. For example, Figure 1 Host 105 may include some updates to be returned in iteration request 625. If this number is less than the number of updates stored in log block 325, key logger 545 may know that not all information has been read from log block 325 and sent to host 105. This example demonstrates another scenario where log block 325 can still store updates even after iteration request 630.

[0098] In another embodiment of the present invention, Figure 1 The host 105 may include components from... Figure 1The host 105 receives the identifier of the last information or some updates to be deleted as parameters of the log deletion request 630. Then, the key logger 545 can delete information up to the specified update from the log block 325.

[0099] Note that in order to know which information to delete from log block 325 and which information to retain in log block 325, the key logger 545 may need more than just keys. For example, consider the case where log block 325 stores information about three requests: the first request writes the value of the key marked k1, the second request writes the value of the key marked k2, and the third request deletes the value of the key marked k1, and assumes... Figure 1 Host 105 sends a log deletion request 630, but only indicates the key marked k1 as the last key to be deleted. If key logger 545 only deletes the information from log block 325 up to the first request involving the key marked k1, the second and third requests will remain in log block 325. Figure 1 Host 105 has already received information about these two requests, so leaving them in log block 325 will result in... Figure 1 Host 105 receives the same information multiple times in different iterative requests 625. On the other hand, if Figure 1 If host 105 only receives information about the first request, then deleting information about the second and third requests would mean... Figure 1 Host 105 will not receive information about these requests. In either case, Figure 5 The secondary index structure 535 may eventually become inconsistent with the primary index structure 530. Therefore, the key recorder 545 may need to come from... Figure 1 More information about host 105, not just the key, and log deletion request 630 can include, for example, information about... Figure 3 Operations 330-1, 330-2, or 330-3 or Figure 3 Information from options 340-1, 340-2, or 340-3, which can identify iterations to Figure 1 The last update of host 105, or the identifier of an entry in log block 325 (such as an entry number, where entries in log block 325 are enumerated starting from the first entry in log block 325).

[0100] As described above, in some embodiments of the present invention, the key logger 545 can store the log block 325 in a dedicated memory, which is Figure 1 Host 105 is inaccessible via a key-value request. In such an embodiment of the present invention, log deletion request 630 can be used to delete information in log block 325. However, in an embodiment of the present invention, log block 325 may be stored as key-value pairs (similar to user data). Figure 5 Among the flash memory chips 515-1 to 515-8, Figure 1 Host 105 can delete log block 325 by issuing deletion request 610. In such an embodiment of the present invention, log deletion request 630 can be omitted to support deletion request 610.

[0101] Figure 7 An alternative view is shown of the operations involved in executing a merge key according to an embodiment of the present invention. Figure 7 In the process, host 105 can send an iteration request 625 to key-value storage device 125. In the response 705, key-value storage device 125 can retrieve... Figure 3 Log block 325 returns information. As described above, iterative request 625 and response 705 can be executed multiple times as appropriate. Then, host 105 can extract the key from the information returned in response 705, as shown in operation 710.

[0102] Then, host 105 can send read request 715 to read. Figure 5 The secondary index structure is 535, the key-value storage device is 125, and the response time is 720. Figure 5 The secondary index structure 535. Operations 715 and 720 can be performed if the host 105 has not yet stored the secondary index structure 535: for example, if a power outage causes... Figure 1 host 105 Figure 1 The memory 115 Figure 5 The secondary index structure 535 is lost. (As described above, stored on key-value storage device 125) Figure 5 The secondary index structure 535 can avoid recreating it from scratch in this case. Figure 5 The secondary index structure 535 is required. (If host 105 already has...) Figure 5 If a copy of the secondary index structure 535 is obtained, operations 715 and 720 can be omitted.

[0103] Then, host 105 can... Figure 5 The secondary index structure 535 is updated using operation 725. This update operation 725 may involve performing a merge sort operation to... Figure 5 Add a key to the secondary index structure 535, such as Figure 3 The update of log block 325 is shown (and from) Figure 5 The secondary index structure 535 deletes the key, such as Figure 3 (As shown in the update of log block 325), and produces a sorted structure.

[0104] Finally, host 105 can send write request 730 to key-value storage device 125 to... Figure 5The secondary index structure 535 is written (again, to avoid host 105 needing to be recreated from scratch). Figure 5 (Secondary index structure 535). Write request 730 is optional because key-value storage device 125 may not need to store it. Figure 5 The secondary index structure 535. Finally, host 105 can send a log deletion request 720 to key-value storage device 125 to retrieve logs from... Figure 3 Log block 325 was deleted and updated.

[0105] Figure 8 An embodiment based on the concept of the present invention is shown. Figure 1 The processor 110 will merge the keys into Figure 5 In the secondary index structure 535. Figure 8 In this process, processor 110 can receive log block 325 (or more precisely, receive updates from log block 325). Processor 110 can then use these updates to merge sort keys into the second secondary index structure 805 to produce a new secondary index structure 535.

[0106] In some embodiments of the present invention, the cooperative key ordering can be determined by... Figure 1 The key-value storage device 125 executes key records to perform the operation, while Figure 1 Host 105 can perform updates to secondary index structure 535. This combination utilizes... Figure 1 The key-value storage device 125 has the capability to track the data that needs to be stored. Figure 5 What changes were made to the secondary index structure 535, and how were they utilized? Figure 1 The host 105's processing power is used to update the secondary index structure 535 to avoid the key-value storage device 125 being updated. Figure 5 The main index structure 530 temporarily blocks I / O. However, since processor 110 can be any type of processor, therefore Figure 1 The key-value storage device 125 should include a processor, which can be used to send key-value pairs to the storage device 125. Figure 1 The key-value storage device 125 internally generates a secondary index structure 535, without involving Figure 1 Host 105. Therefore, if Figure 1 The key-value storage device 125 includes a processor, which may be a CPU, GPU, GPGPU, appropriately configured FPGA, appropriately designed ASIC, or any other type of processor, not... Figure 1 The host 105 can be responsible for generating the secondary index structure 535. In other words, embodiments of the present invention may include any of the functions described above and / or below, which are provided by... Figure 1 It was executed by host 105, not by... Figure 1 The processor of the key-value storage device 125 performs (in order to avoid) Figure 1 Other processing of the key-value storage device 125 is delayed. In some embodiments of the present invention, the processor that can generate and / or update the secondary index structure 535 can be decoupled from any other processor, such as... Figure 5 The processor used in the SSD controller 510 is likely... Figure 1 (Part of the key-value storage device 125.)

[0107] Figure 9 An embodiment of the invention is shown for use in Figure 1 A flowchart illustrating an example process for executing key recording on storage device 125. Figure 9 In block 905, Figure 5 The key-value command handler 540 can be obtained from Figure 1 Host 105 receives requests, such as Figure 6 Write request 605 or Figure 6 Deletion request 610. In block 910, Figure 5 The key-value command handler 540 can handle this request. In block 915, Figure 5 The key-value command handler 540 can be updated. Figure 5 The main index structure 530 is used to reflect any changes caused by requests.

[0108] In block 920, Figure 5 The keylogger 545 can record keys and other information in Figure 3 In log block 325. (See above reference.) Figure 3 The other information discussed may include Figure 3 Operation codes 330-1, 330-2, and 330-3 and / or Figure 3 Options 340-1, 340-2, and 340-3. Additionally, Figure 5 The key logger 545 can use the requested information, such as key group metadata, when logging keys and other information. Finally, in block 925, Figure 1 The key-value storage device 125 can return the result of the request to Figure 1 Host 105.

[0109] Figure 10 An embodiment of the invention is shown for use in Figure 5 The flowchart illustrates an example process by which a key logger records keys using key group metadata. Figure 10 In block 1005, Figure 5 The key logger 545 can determine whether the request includes any key group metadata. If so, then in block 1010, Figure 5 The key logger 545 can log information about the request associated with the key group in Figure 3In log block 325. Otherwise, in block 1015, Figure 5 The key logger 545 can record information about the request in Figure 3 In log block 325, it is generic (i.e., independent of a specific key group).

[0110] Figure 11A-11B An embodiment of the invention is shown for use in Figure 1 Storage device 125 helps Figure 1 The processor 110 will merge the keys into Figure 5 The flowchart of the example process in the secondary index structure 535. Figure 11A In block 1105, Figure 5 The key-value command handler 540 can be obtained from Figure 1 Host 105 receives Figure 6 Iteration request 625. In block 1110, Figure 5 The key logger 545 can determine Figure 3 Is log block 325 stored as an object (i.e., a key-value pair)? Figure 1 In the main memory of key-value storage device 125. If so, then in block 1115, Figure 5 The key recorder 545 can be used with Figure 3 The key associated with log block 325 is determined to be a key-value pair, and in block 1120, Figure 5 The key logger 545 can return the key to Figure 1 Host 105. If in Figure 1 The key-value storage device 125 contains objects stored in its main memory. Figure 3 Multiple log blocks 325, 1115, and 1120 can be used to identify the key for each such log block object. Alternatively, Figure 5 The key recorder 545 can simply return the key as an object stored in the database. Figure 1 The key-value storage device 125 in the main memory Figure 3 A log block with key 325, and let Figure 1 Host 105 repeatedly iterates through request 625 (block 1105) for each such log block.

[0111] On the other hand, if Figure 3 Log block 325 was not stored as an object. Figure 1 The key-value storage device 125 is stored in the main storage (in other words, the log block 325 is stored in the main storage). Figure 1 Somewhere outside the main memory of the key-value storage device 125, such as Figure 5 The additional storage elements 550-1 and / or 550-2 may therefore not be able to be used by Figure 1If the host 105 is accessed directly, then in block 1125, Figure 5 The key logger 545 can access information stored in the database. Figure 3 The information about the operation is in log block 325, and in block 1130, Figure 5 The key logger 545 can return such information to Figure 1 Host 105. If it exists. Figure 3 More than one log block 325 can be used for Figure 3 Each log block 325 repeats blocks 1125 and 1130, or Figure 1 The host 105 can send multiple times. Figure 6 Iteration request 625 to receive information about Figure 1 All information regarding requests to update key values ​​in key-value storage device 125.

[0112] In another embodiment of the inventive concept, even if Figure 3 Log block 325 can be stored as an object. Figure 1 In the main memory of key-value storage device 125, Figure 5 The key-value logger 545 can also continue executing blocks 1125 and 1130. Therefore, even Figure 3 Log block 325 is stored as an object. Figure 1 In key-value storage device 125, Figure 5 The key recorder 545 can still iterate from Figure 3 The information in log block 325, instead of just providing information related to... Figure 3 The key associated with log block 325 and let Figure 1 Host 105 reads storage Figure 3 The object in log block 325.

[0113] regardless Figure 5 The key recorder 545 is used to record information from... Figure 3 The update information for log block 325 is returned directly to Figure 1 The host 105, or Figure 5 The key recorder 545 to Figure 1 The host 105 provides identification Figure 3 Log block 325 is stored Figure 1 The keys of objects in the main storage of key-value storage device 125, in block 1135 ( Figure 11B ), Figure 5 The key-value command handler 540 can receive log deletion requests 630. (Note that in...) Figure 3 Log block 325 is stored as an object. Figure 1 In the case of the main storage of key-value storage device 125, log deletion request 630 can be Figure 6Deletion request 610, because Figure 3 Log block 325 can be stored as if it were stored in Figure 1 (The key-value storage device 125 is deleted just like any other object in its main memory). Then, in block 1140, Figure 1 The key-value storage device 125 can be deleted. Figure 3 Log block 325 (or being deleted) Figure 1 Stored in the main memory of key-value storage device 125 Figure 3 The object of log block 325, or through Figure 5 Key-value logger 545 from Figure 3 Log block 325 deletes and updates, as appropriate how to store... Figure 3 (Log block 325).

[0114] Figure 12 An embodiment of the invention is shown. Figure 1 The processor 110 will merge the keys into Figure 5 The flowchart of the example process in the secondary index structure 535. Figure 12 In block 1205, Figure 1 The host 105 can be accessed from Figure 1 The key-value storage device 125 request comes from Figure 3 The key (and other information) in log block 325, and in block 1210, Figure 1 The host 105 can be accessed from Figure 1 The key-value storage device 125 receives the key (and other information). Figure 1 The host 105 can be used Figure 6 The iterative command 625 executes the request.

[0115] In block 1215, Figure 5 Host 105 can be used from Figure 1 The key-value storage device 125 receives keys (and other information) to generate Figure 5 The secondary index structure is 535. Figure 1 The host 105 can send to Figure 1 Send key-value storage device 125 Figure 6 Write request 605 to Figure 5 The secondary index structure 535 is written. Finally, in block 1225, Figure 1 The host 105 can send to Figure 1 Send key-value storage device 125 Figure 6 Log deletion request 630 to delete Figure 3 Log block 325.

[0116] As described above, in some embodiments of the present invention, Figure 1The key-value storage device 125 can perform Figure 5 The update of the secondary index structure 535, especially when Figure 1 The key-value storage device 125 may include Figure 1 The processor 110 (and memory 115) (or its own processor 110 and / or Figure 1 The memory 115) is used to prevent interference Figure 1 When performing updates in the case of other operations (especially I / O operations) of the key-value storage device 125. Figure 1 A portion of the key-value storage device 125 can be used Figure 12 When the example process is outlined in the document, Figure 1 The processor 110 can also utilize its in certain ways Figure 1 The presence of key-value storage device 125. Figure 14 An embodiment of the invention is shown. Figure 1 Key-value storage devices will merge keys into Figure 5 The flowchart shows an example process in the secondary index structure.

[0117] Figure 13 An embodiment of the invention is shown for use in Figure 1 The processor 110 from Figure 3 A flowchart of an example procedure for accessing the log block 325 access key. Specifically, Figure 13 It shows when Figure 3 Log block 325 is stored in Figure 1 When the key-value storage device 125 is in the main memory, Figure 1 The processor 110 from Figure 3 A flowchart illustrating an example procedure for accessing the 325 access key in log block. Figure 13 middle, Figure 1 The host 105 can be accessed from Figure 1 The key-value storage device 125 receives and identifies the key-value storage device stored in the key-value storage device 125. Figure 1 The keys of objects in the main memory of key-value storage device 125. In block 1310, Figure 1 The host 105 can use the received key to send a read request to the key-value storage device 125. In block 1315, in response to the read request, Figure 1 The host 105 can receive from the key-value storage device 125 Figure 3 Log block 325. Finally, at block 1320, Figure 1 Host 105 can access from Figure 1 The key-value storage device 125 receives data from... Figure 3 The key (and other information) of log block 325.

[0118] Figure 14 Similar to Figure 12 But when Figure 1 The processor 110 includes Figure 1 When in key-value storage device 125, Figure 1 The processor 110 can directly access Figure 3 Log block 325, instead of from Figure 1 The key-value storage device 125 requests a key, whether stored in Figure 1 Is it stored in the main memory of key-value storage device 125, or in... Figure 5 In the additional storage elements 550-1 and / or 550-2 ( Figure 1 Host 105 may be inaccessible. Therefore, in Figure 14 Block 1405, Figure 1 The processor 110 can access Figure 3 Log block 325, and in block 1410, Figure 1 The processor 110 can be from Figure 3 Read the key (and other information) from log block 325. Figure 14 The rest (blocks 1215, 1220, and 1225) can be made by Figure 1 The processor 110 in Figure 1 Executed within the key-value storage device 125, and by Figure 12 The host 105 performs the same action, such as Figure 12 The flowchart of an example process is shown. When included Figure 1 When in the key-value storage device 125, it can also be used by Figure 1 The processor 110 executes the following Figure 15-16 Regardless of whether it is explicitly described.

[0119] Figure 15 An embodiment of the invention is shown. Figure 1 The processor 110 will key and Figure 8 The flowchart shows an example process for merging the second secondary index structure 805. Figure 15 In block 1505, Figure 1 The processor 110 can be from Figure 1 Key-value storage device 125 read Figure 8 The second secondary index structure is 805. In block 1510, Figure 1 The processor 110 can perform a merge sort operation to extract from... Figure 3 The key (and other information) read from log block 325 is merged into... Figure 8 In the second secondary index structure 805, to generate Figure 5 The second secondary index structure is 535.

[0120] Figure 16 An embodiment of the invention is shown. Figure 1 Processor 110 requests deletion Figure 3 A flowchart of the example process for log block 325. Figure 16 In block 1605, Figure 1 The processor 110 can send Figure 6 Log deletion request 630, request Figure 5 Deleting key logger 545 Figure 3 Log block 325. Or (for example, if) Figure 3 Log block 325 is stored in Figure 1 In the main memory of key-value storage device 125, Figure 1 The processor 110 can send Figure 6 Request 610 to delete Figure 3 Log block 325. In block 1605, Figure 1 The processor 110 can send deletion requests for keys associated with log blocks.

[0121] exist Figure 16 What is not shown is that when Figure 1 The processor 110 includes Figure 1 When in the key-value storage device 125, it can be used Figure 1 If the host is unavailable (e.g., 105), delete it directly using other methods. Figure 3 Log block 325 (i.e., used only) Figure 1 The key-value storage device 125 has internal functionalities, such as the ability to directly delete data from additional storage elements 550-1 and / or 550-2. Embodiments of the inventive concept may include deletion using any other means. Figure 3 Log block 325 to delete Figure 3 Log block 325.

[0122] Figures 9-16 Some embodiments of the inventive concept are illustrated in the diagram. However, those skilled in the art will recognize that other embodiments of the inventive concept can be implemented by changing the order of blocks, by omitting blocks, or by including links not shown in the diagram, without regard to any elements that can be specifically omitted. All such variations of the flowchart, whether explicitly described or not, are considered embodiments of the inventive concept.

[0123] Embodiments of this invention offer technical advantages over some implementations. By separating the key record and merging processes, the key-value store can support key records while leaving the merging process to the host. Furthermore, since the result of the merging process is a secondary index structure, the primary index structure used to manage key-value pairs in the key-value store is unaffected by the merging process. This fact means that the key-value store can continue to use the primary index structure for input / output purposes, even if the secondary index structure is being updated. By avoiding the need for the key-value store to interrupt or otherwise delay input / output processing, the key-value store can provide improved performance. However, the secondary index structure can still be used to satisfy range queries.

[0124] The following discussion is intended to provide a brief, general description of one or more suitable machines in which certain aspects of the inventive concept can be implemented. At least in part, a machine or multiple machines can be controlled by input from input devices such as a keyboard, mouse, etc., and by instructions received from another machine, interaction with a virtual reality (VR) environment, biometric feedback, or other input signals. As used herein, the term "machine" broadly refers to a system that includes a single machine, a virtual machine, or communication-coupled machines, virtual machines, or devices operating together. Exemplary machines include computing devices such as personal computers, workstations, servers, portable computers, handheld devices, telephones, tablets, etc., and transportation devices such as private or public transportation (e.g., cars, trains, taxis, etc.).

[0125] The machine or multiple machines may include embedded controllers, such as programmable or non-programmable logic devices or arrays, application-specific integrated circuits (ASICs), embedded computers, smart cards, etc. The machine or multiple machines may utilize one or more connections to one or more remote machines, such as via network interfaces, modems, or other communication coupling. The machines may be interconnected via physical and / or logical networks (such as intranets, the Internet, local area networks, wide area networks, etc.). Those skilled in the art will understand that network communications can utilize various wired and / or wireless short-range or long-range carriers and protocols, including radio frequency (RF), satellite, microwave, Institute of Electrical and Electronics Engineers (IEEE) 802.11, etc. Optics, infrared, cables, lasers, etc.

[0126] Embodiments of this invention can be described by reference to or in conjunction with associated data, including functions, programs, data structures, applications, etc., which, when accessed by a machine, cause the machine to perform tasks or define abstract data types or low-level hardware contexts. The associated data can be stored, for example, in volatile and / or non-volatile memory (e.g., RAM, ROM, etc.), or in other storage devices and their associated storage media, including hard disk drives, floppy disks, optical storage, magnetic tape, flash memory, memory sticks, digital video disks, bio-memory disks, etc. The associated data can be transmitted over a transmission environment (including physical and / or logical networks) in the form of packets, serial data, parallel data, propagated signals, etc., and can be used in compressed or encrypted formats. The associated data can be used in a distributed environment and can be stored locally and / or remotely for machine access.

[0127] Embodiments of the inventive concept may include a tangible, non-transitory machine-readable medium comprising instructions executable by one or more processors, including instructions for performing elements of the inventive concept as described herein.

[0128] Having described and illustrated the principles of the inventive concept with reference to the illustrated embodiments, it will be appreciated that the illustrated embodiments can be modified in arrangement and detail without departing from these principles, and can be combined in any desired manner. Furthermore, although the foregoing discussion focuses on specific embodiments, other configurations are considered. In particular, even when expressions such as "an embodiment of the inventive concept" are used herein, these phrases refer generally to the possibility of embodiments and do not imply limitation of the inventive concept to the specific embodiment configuration. As used herein, these terms may refer to the same or different embodiments that can be combined into other embodiments.

[0129] The illustrative embodiments described above should not be construed as limiting the inventive concept. Although some embodiments have been described, those skilled in the art will readily understand that many modifications can be made to these embodiments without substantially departing from the novel teachings and advantages of this disclosure. Therefore, all such modifications are intended to be included within the scope of the inventive concept as defined in the claims.

[0130] The embodiments of this disclosure can be extended to, but are not limited to, the following statements:

[0131] Statement 1. Embodiments of this invention include a key-value (KV) storage device, comprising:

[0132] A first storage device is used to store data, the first storage device including persistent storage;

[0133] A second storage unit is used to store a main index structure, wherein the main index structure maps a first key to a first position in the first storage unit that stores a value, and the first key and the value form a key-value pair.

[0134] The controller uses the first storage to process at least one of a read request, a write request, or a delete request received from the host; and

[0135] A third storage unit is used to store a secondary index structure that stores the first key, wherein the secondary index structure is sorted.

[0136] Statement 2. One embodiment of the present invention includes the KV storage device according to Statement 1, wherein the KV storage device includes a KV solid-state drive (SSD).

[0137] Statement 3. One embodiment of the present invention includes the KV storage device according to Statement 1, wherein a third storage unit maps a key to a first location in a first storage unit that stores the value.

[0138] Statement 4. One embodiment of the present invention includes the KV storage device as described in Statement 1, wherein the KV storage device can use a first secondary index structure to respond to range queries.

[0139] Statement 5. One embodiment of the present invention includes the KV storage device according to Statement 1, wherein the first secondary index structure includes at least one of a B+-tree, a B-tree, a time-slot page, or a skip list.

[0140] Statement 6. One embodiment of the present invention includes the KV storage device according to Statement 1, wherein the first storage device includes at least one of a second memory and a third memory.

[0141] Statement 7. One embodiment of the present invention includes the KV storage device according to Statement 1, and further includes a fourth storage unit that stores log blocks, the log blocks including a second key, the second key and a second location being stored in the main index structure.

[0142] Statement 8. One embodiment of the present invention includes the KV storage device according to Statement 7, wherein the first secondary index structure does not include the second key.

[0143] Statement 9. One embodiment of the present invention includes the KV storage device according to Statement 7, wherein the fourth storage includes at least one of non-volatile memory and battery-supported volatile memory.

[0144] Statement 10. One embodiment of the present invention includes the KV storage device according to Statement 7, wherein the fourth storage includes at least one of block memory, key-value memory, or byte-addressable memory.

[0145] Statement 11. One embodiment of the present invention includes the KV storage device according to Statement 7, wherein the first storage device includes the fourth storage device.

[0146] Statement 12. One embodiment of the present invention includes the KV storage device according to Statement 7, wherein the log block further includes at least one of an operation identifier and metadata.

[0147] Statement 13. One embodiment of the present invention includes the KV storage device according to Statement 7, wherein the log block is associated with one of at least two log blocks, each of the at least two log blocks being associated with a key group identifier.

[0148] Statement 14. One embodiment of the present invention includes the KV storage device according to Statement 13, wherein at least one of the write request or the delete request may include a key group identifier as metadata.

[0149] Statement 15. One embodiment of the present invention includes the KV storage device according to Statement 7, and further includes a key logger for adding the second key to the log block based at least in part on at least one of the write request or the delete request from the host, including at least one of the write request or the delete request for the second key.

[0150] Statement 16. One embodiment of the present invention includes the KV storage device according to Statement 15, wherein the key recorder sends the second key to the host at least in part based on receiving an iteration request from the host.

[0151] Statement 17. One embodiment of the present invention includes the KV storage device according to Statement 15, wherein the key logger sends a third key associated with the log block.

[0152] Statement 18. One embodiment of the present invention includes the KV storage device as described in Statement 17, wherein the host can use the read request to request the log block.

[0153] Statement 19. One embodiment of the present invention includes the KV storage device according to Statement 15, wherein the key logger deletes the log block at least in part based on receiving a log deletion request from the host.

[0154] Statement 20. One embodiment of the present invention includes the KV storage device as described in Statement 15, and further includes a processor that generates the first secondary index structure from the log block.

[0155] Statement 21. One embodiment of the present invention includes the KV storage device according to Statement 20, wherein the processor includes at least one of a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose GPU (GPGPU), a field-programmable gate array (FPGA), and an application-specific integrated circuit (ASIC).

[0156] Statement 22. One embodiment of the present invention includes the KV storage device according to Statement 20, wherein the processor generates the first secondary index structure and the second secondary index structure from the log block.

[0157] Statement 23. One embodiment of the present invention includes the KV storage device according to Statement 22, wherein the processor performs a merge sort algorithm on the second key and data in the second secondary index structure.

[0158] Statement 24. One embodiment of the present invention includes the KV storage device according to Statement 15, wherein the key logger adds the second location to the log block at least in part based on the write request from the host.

[0159] Statement 25. One embodiment of the present invention includes the KV storage device according to Statement 24, wherein the key recorder sends the second location to the host at least in part based on receiving an iteration request from the host.

[0160] Statement 26. One embodiment of the present invention includes a method comprising:

[0161] Receive a request from the host at the key-value (KV) storage device. The request includes at least an operation and a first key. The operation includes at least one of a write operation or a delete operation.

[0162] The request is processed using a first storage unit of the KV storage device, based at least in part on the operation and the first key, thereby determining the location in the first storage unit associated with the first key;

[0163] The primary index structure of the KV storage device is updated at least in part based on the operation, the first key, and the position;

[0164] The operation and the first key are recorded in the log block of the KV storage device; and

[0165] The result of the request is returned from the KV storage device to the host.

[0166] Statement 27. One embodiment of the present invention includes the method according to Statement 26, wherein recording the operation in a log block of the KV storage device and the first key includes recording the position in the log block of the KV storage device.

[0167] Statement 28. One embodiment of the inventive concept includes the method according to Statement 26, wherein the KV storage device includes a KV solid-state drive (SSD).

[0168] Statement 29. One embodiment of the present invention includes the method according to Statement 26, wherein, at least in part based on the operation, the first key, and the location, the request is received from the host at the KV storage device, the request is processed using the first memory of the KV storage device, the main index structure of the KV storage device is updated, and the operation and the first key in the log block of the KV storage device are recorded as atomic operations.

[0169] Statement 30. One embodiment of the inventive concept includes the method according to Statement 26, wherein:

[0170] The request also includes key group metadata; and

[0171] Recording the operation and the first key in the log block of the KV storage device includes recording the operation and the first key in at least one of at least two log blocks of the KV storage device, based at least in part on the key group metadata.

[0172] Statement 31. One embodiment of the inventive concept includes the method according to Statement 26, further comprising:

[0173] Receive an iteration request, wherein the iteration request is different from a read request;

[0174] Read the first key from the log block; and

[0175] Return to the first key.

[0176] Statement 32. One embodiment of the inventive concept includes the method according to Statement 31, wherein:

[0177] Reading the first key from the log block includes reading the position from the log block; and

[0178] Returning to the first key includes returning to the stated location.

[0179] Statement 33. One embodiment of the inventive concept includes the method according to Statement 31, further comprising:

[0180] Receive a log deletion request, wherein the log deletion request is different from a deletion request; and

[0181] Delete the log block from the KV storage device.

[0182] Statement 34. One embodiment of the inventive concept includes the method according to Statement 26, further comprising:

[0183] Receive an iteration request, wherein the iteration request is different from a read request;

[0184] Identify the second key associated with the log block; and

[0185] Return to the second key.

[0186] Statement 35. One embodiment of the present invention includes a method comprising:

[0187] Request the first key from the log block on the key-value (KV) storage device;

[0188] Receive the first key from the KV storage device; and

[0189] A first secondary index structure is generated at least partially based on the first key, and the first secondary index structure is sorted at least according to the first key and the second key.

[0190] The first secondary index structure is different from the primary index structure used by the KV storage device to process read requests including the first key, write requests including the first key, or delete requests including the first key.

[0191] Statement 36. One embodiment of the inventive concept includes the method according to Statement 35, wherein:

[0192] Requesting the first key from the log block on the KV storage device includes requesting the location from the log block on the KV storage device;

[0193] Receiving the first key from the KV storage device includes receiving the location from the KV storage device; and

[0194] Generating a first secondary index structure based at least in part on the first key includes generating the first secondary index structure based at least in part on the first key and the position.

[0195] Statement 37. One embodiment of the inventive concept includes the method according to Statement 35, wherein the KV storage device includes a KV solid-state drive (SSD).

[0196] Statement 38. One embodiment of the inventive concept includes the method according to Statement 35, wherein the first secondary index structure includes at least one of a B+-tree, a B-tree, a time-slot page, or a skip list.

[0197] Statement 39. One embodiment of the inventive concept includes the method according to Statement 35, wherein:

[0198] Requesting the key from the log block on the KV storage device includes the host requesting the first key from the log block on the KV storage device;

[0199] Receiving the first key from the KV storage device includes receiving the first key from the KV storage device at the host; and

[0200] Generating the first secondary index structure includes generating the first secondary index structure on the host at least in part based on the first key.

[0201] Statement 40. One embodiment of the inventive concept includes the method according to Statement 39, wherein receiving the first key from the KV storage device at the host comprises:

[0202] Receive a second key associated with the log block at the host from the KV storage device;

[0203] The host requests the log block from the KV storage device using the second key;

[0204] Receive the log block from the host at the KV storage device; and

[0205] The host accesses the first key from the log block.

[0206] Statement 41. One embodiment of the inventive concept includes the method according to Statement 35, wherein:

[0207] Requesting the first key from the log block on the KV storage device includes the processor of the KV storage device requesting the first key from the log block on the KV storage device;

[0208] Receiving the first key from the KV storage device includes receiving the first key at the processor of the KV storage device; and

[0209] Generating the first secondary index structure includes generating the first secondary index structure on the processor of the KV storage device, at least in part, based on the first key.

[0210] Statement 42. One embodiment of the inventive concept includes the method according to Statement 41, wherein:

[0211] The processor of the KV storage device requests the first key from the log block on the KV storage device, which includes accessing the location of the log block on the KV storage device;

[0212] Receiving the first key at the processor of the KV storage device includes reading the first key from the location of the log block on the KV storage device.

[0213] Statement 43. One embodiment of the present invention includes the method according to Statement 35, wherein generating the first secondary index structure based at least in part on the first key includes merging the first key with a second secondary index structure.

[0214] Statement 44. One embodiment of the present invention includes the method according to Statement 43, wherein the second secondary index structure is stored in memory.

[0215] Statement 45. One embodiment of the present invention includes the method according to Statement 44, wherein generating the first secondary index structure based at least in part on the first key further includes reading a second secondary index structure from the KV storage device into the memory.

[0216] Statement 46. One embodiment of the inventive concept includes the method according to Statement 43, generating the first secondary index structure at least in part based on the first key, and further includes requesting the deletion of the log block from the KV storage device.

[0217] Statement 47. One embodiment of the present invention includes the method according to Statement 46, wherein requesting the deletion of the log block from the KV storage device includes sending a request from a host to the KV storage device to delete the log block from the KV storage device.

[0218] Statement 48. One embodiment of the present invention includes the method according to Statement 46, wherein requesting the log block to be deleted from the KV storage device includes deleting the log block from the KV storage device.

[0219] Statement 49. One embodiment of the present invention includes the method according to Statement 43, generating the first secondary index structure at least in part based on the first key, and further includes writing the first secondary index structure into the KV storage device.

[0220] Statement 50. One embodiment of the present invention includes an article comprising a non-transitory storage medium storing instructions thereon, which, when executed by a machine, cause:

[0221] A request is received from the host at the key-value (KV) storage device, the request including at least one operation and a first key, the operation including at least one of a write operation or a delete operation;

[0222] The request is processed using a first storage unit of the KV storage device, at least in part based on the operation and the first key, thereby determining the location in the first storage unit associated with the first key;

[0223] The primary index structure of the KV storage device is updated at least in part based on the operation, the first key, and the position;

[0224] The operation and the first key are recorded in the log block of the KV storage device; and

[0225] The result of the request is returned from the KV storage device to the host.

[0226] Statement 51. One embodiment of the present invention includes the article according to Statement 50, wherein the operation recorded in the log block of the KV storage device and the first key include recording the position in the log block of the KV storage device.

[0227] Statement 52. One embodiment of the inventive concept includes the article according to Statement 50, wherein the KV storage device includes a KV solid-state drive (SSD).

[0228] Statement 53. One embodiment of the present invention includes an article according to Statement 50, wherein, at least in part based on the operation, the first key, and the location, a request is received from the host at the KV storage device, the request is processed using the first memory of the KV storage device, the main index structure of the KV storage device, the first key, and the location are updated, and the operation and the first key are recorded in the log block of the KV storage device as atomic operations.

[0229] Statement 54. One embodiment of this disclosure includes the article described in Statement 50, wherein:

[0230] The request also includes key group metadata; and

[0231] Recording the operation and the first key in the log block of the KV storage device includes recording the operation and the first key in at least one of at least two log blocks of the KV storage device, based at least in part on the key group metadata.

[0232] Statement 55. One embodiment of the inventive concept includes the article according to Statement 50, wherein a non-transitory storage medium further stores instructions thereon that, when executed by a machine, cause:

[0233] Receive an iteration request, wherein the iteration request is different from a read request;

[0234] Read the first key from the log block; and

[0235] Return to the first key.

[0236] Statement 56. One embodiment of the inventive concept includes the article described in statement 55, wherein:

[0237] Reading the first key from the log block includes reading the position from the log block; and

[0238] Returning to the first key includes returning to the stated location.

[0239] Statement 57. One embodiment of the inventive concept includes the article according to Statement 55, wherein a non-transitory storage medium further stores instructions thereon that, when executed by a machine, cause:

[0240] Receive a log deletion request, wherein the log deletion request is different from a deletion request; and

[0241] Delete the log block from the KV storage device.

[0242] Statement 58. One embodiment of the inventive concept includes the article according to Statement 50, wherein a non-transitory storage medium further stores instructions thereon that, when executed by a machine, cause:

[0243] Receive an iteration request, wherein the iteration request is different from a read request;

[0244] Identify the second key associated with the log block; and

[0245] Return to the second key.

[0246] Statement 59. One embodiment of the present invention includes an article comprising a non-transitory storage medium storing instructions thereon, which, when executed by a machine, cause:

[0247] Request a key from a log block on a key-value (KV) storage device;

[0248] Receive the first key from the KV storage device; and

[0249] A first secondary index structure is generated at least partially based on the first key, and the first secondary index structure is sorted according to at least the first key and the second key.

[0250] The first secondary index structure is different from the primary index structure used by the KV storage device to process read requests including the first key, write requests including the first key, or delete requests including the first key.

[0251] Statement 60. One embodiment of the inventive concept includes the article described in statement 59, wherein:

[0252] Requesting the first key from the log block on the KV storage device includes requesting the location from the log block on the KV storage device;

[0253] Receiving the first key from the KV storage device includes receiving the location from the KV storage device; and

[0254] Generating a secondary index structure based at least in part on the first key includes generating the first secondary index structure based at least in part on the first key and the position.

[0255] Statement 61. One embodiment of the inventive concept includes the article described in Statement 59, wherein the KV storage device includes a KV solid-state drive (SSD).

[0256] Statement 62. One embodiment of the inventive concept includes the article described in Statement 59, wherein the first secondary index structure includes at least one of a B+-tree, a B-tree, a time-slot page, or a skip list.

[0257] Statement 63. One embodiment of the inventive concept includes the article described in statement 59, wherein:

[0258] Requesting the first key from the log block on the KV storage device includes the host requesting the first key from the log block on the KV storage device;

[0259] Receiving the first key from the KV storage device includes receiving the first key from the KV storage device at the host; and

[0260] Generating the first secondary index structure includes generating the first secondary index structure on the host at least in part based on the first key.

[0261] Statement 64. One embodiment of the inventive concept includes the article according to Statement 63, wherein receiving a first key from the KV storage device at the host comprises:

[0262] Receive a second key associated with the log block at the host machine from the KV storage device;

[0263] The host requests the log block from the KV storage device using the second key;

[0264] Receive the log block from the host at the KV storage device; and

[0265] The host accesses the first key from the log block.

[0266] Statement 65. One embodiment of the inventive concept includes the article described in statement 59, wherein:

[0267] Requesting the first key from the log block on the KV storage device includes the processor of the KV storage device requesting the first key from the log block on the KV storage device;

[0268] Receiving the first key from the KV storage device includes receiving the first key at the processor of the KV storage device; and

[0269] Generating the first secondary index structure includes generating the first secondary index structure on the processor of the KV storage device, at least in part, based on the first key.

[0270] Statement 66. One embodiment of the inventive concept includes the article described in statement 65, wherein:

[0271] The processor of the KV storage device requests the first key from the log block on the KV storage device, which includes accessing the location of the log block on the KV storage device;

[0272] Receiving the first key at the processor of the KV storage device includes reading the first key from the location of the log block on the KV storage device.

[0273] Statement 67. One embodiment of the inventive concept includes the article as described in Statement 59, wherein generating the first secondary index structure based at least in part on the first key includes merging the first key with a second secondary index structure.

[0274] Statement 68. One embodiment of the inventive concept includes the article described in Statement 67, wherein the second secondary index structure is stored in memory.

[0275] Statement 69. One embodiment of the inventive concept includes the article according to Statement 68, wherein generating the first secondary index structure based at least in part on the first key further includes reading the second secondary index structure from the KV storage device into the memory.

[0276] Statement 70. One embodiment of the present invention includes the article as described in Statement 67, generating the first secondary index structure at least in part based on the first key, and also includes requesting that the log block be deleted from the KV storage device.

[0277] Statement 71. One embodiment of the present invention includes the article according to Statement 70, wherein requesting the log block to be deleted from the KV storage device includes sending a request from the host to the KV storage device to delete the log block from the KV storage device.

[0278] Statement 72. One embodiment of the inventive concept includes the article described in Statement 70, wherein requesting the log block to be deleted from the KV storage device includes deleting the log block from the KV storage device.

[0279] Statement 73. One embodiment of the present invention includes the article as described in Statement 67, and generating the first secondary index structure based at least in part on the first key further includes writing the first secondary index structure to the KV storage device.

[0280] Therefore, given the various arrangements of the embodiments described herein, this detailed description and accompanying materials are intended to be illustrative only and should not be considered as limiting the scope of the inventive concept. Thus, the claimed inventive concept comprises all these modifications within the scope and spirit of the following claims and their equivalents.

Claims

1. A key-value (KV) storage device, comprising: A first storage device is used to store data, the first storage device including persistent storage; A second storage unit is used to store a main index structure, wherein the main index structure maps a first key to a first position in the first storage unit that stores a value, and the first key and the value form a key-value pair. A controller for using the first storage to process at least one of a read request, a write request, or a delete request; as well as A third storage unit is used to store a secondary index structure, which stores the first key, wherein the secondary index structure is sorted. The secondary index structure is updated at least in part based on at least one of the read request, the write request, or the delete request, and The KV storage device is configured to receive a range query from the host, identify a second key and a third key in the secondary index structure that satisfy the range query, and send the second key and the third key to the host in response to the range query.

2. The KV storage device according to claim 1 further includes a fourth storage unit, the fourth storage unit storing log blocks, the log blocks including a second key, the second key and a second position being stored in the main index structure.

3. The KV storage device according to claim 2, wherein, The log block also includes at least one of the operation identifier and metadata.

4. The KV storage device of claim 2, further comprising a key logger for adding the second key to the log block based at least in part on at least one of the write request or the delete request, wherein at least one of the write request or the delete request includes the second key.

5. The KV storage device according to claim 4, wherein, The key logger retrieves the second key from the log block and sends the second key at least in part based on the receipt of an iteration request.

6. The KV storage device according to claim 4, wherein, The key logger sends a third key associated with the log block.

7. The KV storage device according to claim 4, wherein, The key logger deletes the log block at least in part based on receiving a log deletion request.

8. The KV storage device according to claim 4 further includes a processor for generating the secondary index structure from the log block.

9. A data storage method, comprising: A request is received at a key-value (KV) storage device, the request including at least an operation and a first key, the operation including at least one of a write operation or a delete operation; The request is processed using the first storage of the KV storage device, at least in part based on the operation and the first key, thereby determining the location in the first storage associated with the first key; The primary index structure of the KV storage device is updated at least in part based on the operation, the first key, and the position; The operation and the first key are recorded in the log block of the KV storage device; Return the result of the request from the KV storage device; The secondary index structure is updated at least in part based on the log blocks; Receive range queries at the KV storage device; Identify the second and third keys in the secondary index structure that satisfy the range query; and The range query from the KV storage device is responded to using the second key and the third key.

10. The method of claim 9, further comprising: Receive an iteration request, wherein the iteration request is different from a read request; Read the first key from the log block; and Return to the first key.

11. The method of claim 10, further comprising: Receive a log deletion request, wherein the log deletion request is different from a deletion request; and Delete the log block from the KV storage device.

12. The method according to claim 9, further comprising: Receive an iteration request, wherein the iteration request is different from a read request; Identify the second key associated with the log block; and Return to the second key.

13. A data storage method, comprising: The host requests the first key from the log block on the key-value storage device; The first key is received from the KV storage device at the host. The host generates a first secondary index structure based at least in part on the first key, and the first secondary index structure is sorted according to at least the first key and the second key; The host sends the first secondary index structure to the KV storage device for storage on the KV storage device; Send a range query from the host to the KV storage device; as well as The host receives a second and a third key from the KV storage device in response to the range query. The first secondary index structure is different from the primary index structure. The primary index structure is used by the KV storage device to process read requests including the first key, write requests including the first key, or delete requests including the first key.

14. The method of claim 13, wherein: The request for a first key by the host from a log block on the KV storage device includes the host requesting the first key from a log block on the KV storage device that is sent to the processor of the KV storage device; as well as Receiving the first key from the KV storage device at the host includes receiving the first key from the processor of the KV storage device at the host.

15. The method according to claim 13, wherein, Generating the first secondary index structure by the host, at least in part, based on the first key, includes merging the first key with a second secondary index structure by the host.

16. The method according to claim 15, wherein, Generating the first secondary index structure by the host, at least in part, based on the first key, also includes the host requesting the deletion of the log block from the KV storage device.

17. The method according to claim 16, wherein, Requesting the deletion of the log block from the KV storage device by the host includes sending a request from the host to the KV storage device to delete the log block from the KV storage device.

18. The method according to claim 16, wherein, The host requests the deletion of the log block from the KV storage device, which includes the KV storage device deleting the log block from the KV storage device.

19. The method according to claim 15, wherein, Generating the first secondary index structure based at least in part on the first key also includes writing the first secondary index structure into the KV storage device.

Images