A data processing method and NVMe storage device

Abstract

A data processing method and a storage device, the data processing method comprising: an NVMe memory receiving an NVMe write command sent by a host, the NVMe write command carrying a key and a value pointer, the value pointer pointing to a first storage space, the first storage space being used to store a value; the NVMe memory obtaining the key and the value length from the NVMe write command, allocating a second storage space for the value according to the value length, the second storage space being in the NVMe memory; the NVMe memory obtaining the value from the host, and storing the value in the second storage space.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of storage, and in particular to the field of NVMe. Background Art

[0002] The NVMe (Non-Volatile Memory Express) protocol is a high-speed interface protocol used in storage systems. The NVMe protocol provides faster read and write speeds and lower latency than the SCSI protocol, and is gaining increasing attention and popularity in the industry.

[0003] With the development of information technology, object storage technology is often used. A common object storage technology is key value (KV) storage. In the prior art, since NVMe devices only support block interfaces, if the host wants to store KV data in NVMe storage, the steps are: the host converts the KV command (generally composed of Key, Value and Metadata) into block data (for example, splitting / merging a KV name into one or at least two block data); the host assigns an LBA address to the block data; the host sends the block data to the NVMe storage; after receiving the block data, the NVMe storage stores the block data one by one according to the assigned LBA address.

[0004] However, in the above steps, converting KV data into block data and assigning LBA addresses to the block data will consume a large amount of host computing resources, resulting in a decrease in storage system performance and affecting the operating efficiency of the host and storage controller. Summary of the invention

[0005] The present invention provides a data processing method, an NVMe memory, and a storage system solution. The efficiency of writing KV data into the NVMe memory can be improved. In some solutions, the efficiency of reading KV data from the NVMe memory can also be improved accordingly.

[0006] According to a first aspect of an embodiment of the present invention, a data processing method is provided, the method comprising: a fast non-volatile storage NVMe memory receives an NVMe write command sent by a host, the NVMe write command carries a key, the NVMe write command carries a value pointer, the value pointer points to a first storage space in the host, the first storage space is used to store a value, and the key and the value belong to the same KV pair; the NVMe memory obtains the key from the NVMe write command, obtains the value length according to the value pointer, allocates a second storage space for the value according to the value length, and the second storage space is in the NVMe memory; the NVMe memory sends a first transmission request to the host, obtains the value from the host, and saves the value in the second storage space. Based on this scheme, in the process of transferring KV data from the host to the NVMe memory, there is no need to convert the KV data into a block form, thereby improving the storage efficiency of the KV data.

[0007] In a first possible implementation of the first aspect, the NVMe memory sends a first transfer request to a host and obtains the value from the host, specifically including: the NVMe memory sends a DMA transfer request to the host, obtains the value from the host, the DMA instruction carries the first storage space as an access address, carries the second storage space as a write address, and the NVMe memory and the host are connected by a PCIe bus. This solution provides a value transfer method using DMA.

[0008] In a second possible implementation of the first aspect, the NVMe memory sends a first transmission request to a host and obtains the value from the host, specifically including: the memory sends an RDMA transmission request to the host and obtains the value from the host, the RDMA instruction carries the first storage space as an access address and carries the second storage space as a write address, wherein the NVMe memory and the host are connected by a Fabric bus. This solution provides a value transmission method using RDMA.

[0009] In a third possible implementation of the first aspect, the NVMe write command further carries a field for the number of KVs, the field for the number of KVs is used to describe the number of KVs in the NVMe write command, and the NVMe memory obtains the same number of keys as the number of KVs from the NVMe write command, and obtains the same number of values ​​as the number of KVs. This solution can support carrying multiple KVs in the same NVMe write command.

[0010] In a fourth possible implementation of the first aspect, wherein: the NVMe write command further carries a field in KV format, wherein the field in KV format describes the structure of the field in the NVMe write command, and the NVMe memory obtains each field from the NVMe write command according to the field content defined by the KV format field. This solution enables the same NVMe device to support NVMe write command messages in multiple formats for transmitting KV, and use one of the formats of the write command message during a specific write operation. Similarly, in the read command process, the same NVMe device can also support NVMe read command messages in multiple formats for transmitting KV.

[0011] In a fifth possible implementation of the first aspect, the NVMe write command also carries the value metadata pointer, and the method further includes: the NVMe memory obtains the metadata length according to the metadata pointer, allocates a fourth storage space for the metadata according to the metadata length, and the fourth storage space is in the NVMe memory; the NVMe memory obtains the metadata from the host through the first transmission request, and stores the metadata in the fourth storage space. This scheme describes a specific scheme for storing metadata in KV.

[0012] In combination with the fifth possible implementation of the first aspect, in a sixth possible implementation, after the metadata transmission fails, the NVMe memory releases the storage space allocated for the value and releases the fourth storage space allocated for the metadata. This solution can release the resources occupied by the value in a timely manner after the metadata transmission fails.

[0013] In combination with the seventh possible implementation of the first aspect, the NVMe memory sends a first transfer request to the host and obtains the value from the host, specifically including one of the following: the memory sends at least two DMA transfer requests to the host to obtain the value, each DMA transfer request is used to request to obtain a part of the value, when any one of the DMA transfer requests fails to execute, the NVMe memory releases the storage space allocated for the value, wherein the NVMe memory and the host are connected by a PCIe bus; or, the memory sends at least two RDMA transfer requests to the host to obtain the value, each RDMA transfer request is used to request to obtain a part of the value, when any one of the RDMA transfer requests fails to execute, the NVMe memory releases the storage space allocated for the value, wherein the NVMe memory and the host are connected by Frabic. This solution can divide the same value into multiple RDMA transfer requests for transmission, thereby reducing the amount of data transmitted by each RDMA.

[0014] In conjunction with an eighth possible implementation of the first aspect, the method further includes: generating a mapping relationship between the key and the second storage space. The mapping relationship can provide a basis for a subsequent KV (especially value) reading solution.

[0015] In combination with the ninth possible implementation of the first aspect, the method further includes: the NVMe memory receives an NVMe read command from the host, and the NVMe read command carries the key. In addition, it may further include: the NVMe memory queries the mapping relationship according to the key to obtain the location information of the second storage space storing the value, and sends the location information of the second storage space to the host; the NVMe memory receives a second transmission request sent by the host, and the second transmission request is used to request to obtain the data stored in the second storage space. The eighth possible implementation of the first aspect can realize the reading of KV (or value).

[0016] In combination with the tenth possible implementation of the first aspect, before the NVMe memory receives the second transmission request sent by the host, it further includes: the host reserves a third storage space in the host according to the size of the second storage space. After the NVMe memory sends the value to the host, it further includes: the host writes the received data from the second storage space into the fourth storage space. This scheme describes the operations performed by the host in the process of reading KV (or value).

[0017] In combination with the eleventh possible implementation of the first aspect, the NVMe read command sent by the host received by the NVMe memory also carries the free space information of the host. After the NVMe memory receives the NVMe read command from the host, it further includes: the NVMe memory determines whether the free storage space of the host is greater than or equal to the second storage space, and if so, executes the step of sending the location information of the second storage space to the host, and if not, ends the step. This scheme carries the free space information of the host in the NVMe read command. The host does not need to determine whether the free space is sufficient and reserve storage space after receiving the response from the NVMe memory, which can reduce the number of interactions between the NVMe memory and the host and improve the efficiency of the host reading the value. It should be noted that the free space here is a space with continuous addresses.

[0018] In combination with a twelfth possible implementation of the first aspect, the first storage space is described by the first address of the first storage space and the length of the value; the second storage space is described by the first address of the second storage space. This scheme describes a description method of the storage space. Relying on the content described by the description method, the storage location of the read value and the storage location of the written value can be located.

[0019] In a second aspect, the present invention provides an implementation of an NVMe memory, including a controller and a storage medium, wherein the controller is connected to the storage medium, the storage medium is used to provide storage space, and the processor is configured to execute: receiving a fast non-volatile storage NVMe write command sent by a host, wherein the NVMe write command carries a key, the NVMe write command carries a value pointer, the value pointer points to a first storage space in the host, the first storage space is used to store a value, and the key and the value belong to the same KV pair; obtaining the key from the NVMe write command, obtaining the value length according to the value pointer, and allocating a second storage space for the value according to the value length, the second storage space being in the storage medium; sending a first transfer request to the host, obtaining the value from the host, and storing the value in the second storage space. The NVMe memory is configured to execute the method of the first aspect and the methods of various implementations of the first aspect.

[0020] The third aspect of the present invention provides an implementation method of a data processing method, the method comprising: a non-volatile storage fast NVMe memory receives an NVMe write command, the header of the NVMe write command carries a key key, the NVMe command also carries a value value, the key corresponds to the value, and the key and the value belong to the same KV pair; the NVMe memory obtains the key and the value from the NVMe write command; the NVMe memory saves the value in a storage medium of the NVMe memory.

[0021] In combination with a first possible implementation of the third aspect, the NVMe write command further carries a field for the number of KVs, and the field for the number of KVs is used to describe the number of KVs in the NVMe write command. The NVMe memory obtains the same number of keys as the number of KVs from the NVMe write command, and obtains the same number of values ​​as the number of KVs.

[0022] In combination with a second possible implementation of the third aspect, the NVMe write command further carries a field in a KV format, wherein the field in the KV format describes the structure of the field in the NVMe write command, and the NVMe memory obtains each field from the NVMe write command according to the field content defined by the KV format field.

[0023] In combination with a third possible implementation of the third aspect, wherein the NVMe write command further carries the length of the key, the NVMe memory obtains the key from the write command specifically including: obtaining the key from the write command from a preset starting position of the key according to the length of the key.

[0024] In combination with a fourth possible implementation of the third aspect, wherein the NVMe write command further carries a value offset and a length of the value, and the NVMe memory obtains the value from the NVMe write command specifically including: the NVMe memory obtains the value from the position indicated by the offset according to the length of the value.

[0025] In combination with a fifth possible implementation of the third aspect, the method further includes: generating a mapping relationship between the key and the value storage space.

[0026] A third possible implementation is a fifth possible implementation based on the third aspect of the present application, and after the method, further includes: the host sends an NVMe read command to the NVMe memory, and the NVMe read command carries the key; the NVMe memory receives the read command from the host, and obtains the key from the NVMe read command; the NVMe memory uses the key to search the value storage space from the mapping relationship; the NVMe memory obtains the value from using the value storage space; the NVMe memory generates a response message to the NVMe read command and sends it to the host, and the response message carries the value.

[0027] In a fourth aspect, the present invention provides an implementation of an NVMe memory, including a controller and a storage medium. The controller is configured to execute the method of the third aspect and various implementation methods of the third aspect.

[0028] In a fifth aspect of an embodiment of the present invention, a storage device is provided, which can be a physical device, such as an NVMe memory; or a logical device, such as a program running in a processor of an NVMe memory, or a program in a storage server. The device includes: an interface module, which is used to receive an NVMe write command sent by a host, wherein the NVMe write command carries a key, and the NVMe write command carries a value pointer, wherein the value pointer points to a first storage space in the host, wherein the first storage space is used to store a value, and the key and the value belong to the same KV pair; a processing module, which is used to obtain the key from the NVMe write command, obtain the value length according to the value pointer, and allocate a second storage space for the value according to the value length; a storage module, which is used to send a first transmission request to the host, obtain the value from the host, and save the value in the second storage space. The second storage space can be provided by a storage medium in the NVMe memory, and the storage medium is connected to the processor of the NVMe memory. Based on this scheme, in the process of transferring KV data from the host to the NVMe memory, it is not necessary to convert the KV data into a block form, thereby improving the storage efficiency of the KV data.

[0029] In a first possible implementation of the fifth aspect, the storage module sends a first transfer request to the host and obtains the value from the host, specifically including: the storage module sends a DMA transfer request to the host, obtains the value from the host, and the DMA instruction carries the first storage space as the access address and carries the second storage space as the write address. The storage device (the storage device is hardware) and the host are connected with a PCIe bus, or the NVMe memory where the storage device (the storage device is software) is located and the host are connected with a PCIe bus. This solution provides a value transfer method using DMA.

[0030] In a second possible implementation of the fifth aspect, the storage module sends a first transmission request to the host and obtains the value from the host, specifically including: the storage module sends an RDMA transmission request to the host, obtains the value from the host, the RDMA instruction carries the first storage space as the access address, and carries the second storage space as the write address, wherein the storage device (the storage device is hardware) and the host are connected by a Fabric bus, or the NVMe storage device (the storage device is software) where the storage device is located and the host are connected by a Fabric bus. This solution provides a value transmission method using RDMA.

[0031] In a third possible implementation of the fifth aspect, the NVMe write command further carries a field for the number of KVs, the field for the number of KVs is used to describe the number of KVs in the NVMe write command, and the NVMe memory obtains the same number of keys as the number of KVs from the NVMe write command, and obtains the same number of values ​​as the number of KVs. This solution can support carrying multiple KVs in the same NVMe write command.

[0032] In a fourth possible implementation of the fifth aspect, wherein: the NVMe write command further carries a field in KV format, wherein the field in KV format describes the structure of the field in the NVMe write command, and the NVMe memory obtains each field from the NVMe write command according to the field content defined by the KV format field. This solution enables the same NVMe device to support NVMe write command messages that transmit KV in multiple formats, and use one of the formats of the write command message during a specific write operation. Similarly, in the read command process, the same NVMe device can also support NVMe read command messages that transmit KV in multiple formats.

[0033] In a fifth possible implementation of the fifth aspect, the NVMe write command also carries the value metadata pointer, and the method further includes: the NVMe memory obtains the metadata length according to the metadata pointer, allocates a fourth storage space for the metadata according to the metadata length, and the fourth storage space is in the NVMe memory; the NVMe memory obtains the metadata from the host through the first transmission request, and stores the metadata in the fourth storage space. This scheme describes a specific scheme for storing metadata in KV.

[0034] In combination with the fifth possible implementation of the fifth aspect, in a sixth possible implementation, after the metadata transmission fails, the processing module is further configured to release the storage space allocated for the value and release the fourth storage space allocated for the metadata. This solution can release the resources occupied by the value in a timely manner after the metadata transmission fails.

[0035] In combination with the seventh possible implementation of the fifth aspect, the storage module sends a first transfer request to the host and obtains the value from the host, specifically including one of the following: the processing module sends at least two DMA transfer requests to the host to obtain the value, each DMA transfer request is used to request to obtain a part of the value, when any one of the DMA transfer requests fails to execute, the storage module releases the storage space allocated for the value, wherein the NVMe memory and the host are connected by a PCIe bus; or, the storage module sends at least two RDMA transfer requests to the host to obtain the value, each RDMA transfer request is used to request to obtain a part of the value, when any one of the RDMA transfer requests fails to execute, the storage module releases the storage space allocated for the value, wherein the NVMe memory and the host are connected by Frabic. This solution can divide the same value into multiple RDMA transfer requests for transmission, reducing the amount of data transmitted by each RDMA.

[0036] In conjunction with an eighth possible implementation of the fifth aspect, the processing module is further configured to generate a mapping relationship between the key and the second storage space, which can provide a basis for a subsequent KV (especially value) reading solution.

[0037] In combination with the ninth possible implementation of the fifth aspect, the interface module is also used to receive an NVMe read command from the host, and the NVMe read command carries the key. The processing module is also used to query the mapping relationship according to the key to obtain the location information of the second storage space storing the value, and the processing module is also used to send the location information of the second storage space to the host through the interface module; the interface module is also used to receive a second transmission request sent by the host, and the second transmission request is used to request to obtain the data stored in the second storage space. The eighth possible implementation of the fifth aspect can realize the reading of KV (or value).

[0038] In combination with the tenth possible implementation of the fifth aspect, before the interface module receives the second transmission request sent by the host, the host is also used to: reserve a third storage space in the host according to the size of the second storage space. After the storage device sends the value to the host, the host is also used to write the received data from the second storage space into the fourth storage space. This scheme describes the function of the host in reading KV (or value).

[0039] In combination with the eleventh possible implementation of the fifth aspect, the NVMe read command sent by the host received by the interface module also carries the free space information of the host. After the NVMe memory receives the NVMe read command from the host, it further includes: the NVMe memory determines whether the free storage space of the host is greater than or equal to the second storage space. If so, executes the step of sending the location information of the second storage space to the host; if not, ends the step. This solution carries the free space information of the host in the NVMe read command. The host does not need to determine whether the free space is sufficient and reserve storage space after receiving the response from the NVMe memory. It can reduce the number of interactions between the NVMe memory and the host, and improves the efficiency of the host reading the value. It should be noted that the free space here is a space with continuous addresses.

[0040] In combination with a twelfth possible implementation of the fifth aspect, the first storage space is described by the first address of the first storage space and the length of the value; the second storage space is described by the first address of the second storage space. This scheme describes a description method of the storage space. Relying on the content described by the description method, the storage location of the read value and the storage location of the written value can be located.

[0041] Aspect 6: In combination with the storage device provided in the fifth aspect and various possible implementations of the fifth aspect, the present invention further provides an implementation of a storage system, which includes a host and a storage device.

[0042] The seventh aspect of the present invention provides a storage device, which can be a physical device, such as an NVMe memory; or a logical device, such as a program running in a processor of an NVMe memory, or a program in a storage server. The device includes: an interface module for receiving an NVMe write command, the header of the NVMe write command carries a key key, the NVMe command also carries a value value, the key corresponds to the value, and the key and the value belong to the same KV pair; a processing module for obtaining the key and the value from the NVMe write command; a storage module for storing the value in a storage medium of the NVMe memory.

[0043] In combination with the first possible implementation of the seventh aspect, the NVMe write command further carries a field for the number of KVs, and the field for the number of KVs is used to describe the number of KVs in the NVMe write command. The processing module obtains the same number of keys as the number of KVs from the NVMe write command, and obtains the same number of values ​​as the number of KVs.

[0044] In combination with a second possible implementation of the seventh aspect, the NVMe write command further carries a field in a KV format, wherein the field in the KV format describes the structure of the field in the NVMe write command, and the processing module obtains each field from the NVMe write command according to the field content defined by the KV format field.

[0045] In combination with a third possible implementation of the seventh aspect, wherein the NVMe write command further carries the length of the key, the processing module obtains the key from the write command specifically including: obtaining the key from the write command from a preset starting position of the key according to the length of the key.

[0046] In combination with a fourth possible implementation of the seventh aspect, wherein the NVMe write command further carries a value offset and a length of the value, the processing module obtains the value from the NVMe write command specifically including: the processing module obtains the value from the position indicated by the offset according to the length of the value.

[0047] In combination with a fifth possible implementation of the seventh aspect, the storage module is further used to: generate a mapping relationship between the key and the value storage space.

[0048] A seventh possible implementation method is based on the fifth possible implementation method of the seventh aspect, and provides a host method: the host is used to send an NVMe read command to the NVMe memory, and the NVMe read command carries the key; the interface module is also used to receive the read command from the host, and obtain the key from the NVMe read command; the processing module is also used to use the key to search the value storage space from the mapping relationship; the processing module is also used to obtain the value from using the value storage space; the processing module is also used to generate a response message to the NVMe read command, and send the response message to the host through the interface module, and the response message carries the value.

[0049] Aspect 8: In combination with the storage device provided in aspect 7 and various possible implementations of aspect 7, the present invention further provides an implementation of a storage system, which includes a host and a storage device.

[0050] Ninth aspect: In combination with the memory provided by the second aspect and various possible implementation methods of the second aspect, the present invention also provides an implementation of a storage system, which includes a host and an NVMe memory.

[0051] Aspect ten: In combination with the memory provided in the fourth aspect and various possible implementations of the fourth aspect, the present invention further provides an implementation of a storage system, which includes a host and a storage device.

[0052] Based on the present invention, a data processing method, an NVMe memory and a storage system solution are provided, which can improve the efficiency of writing KV data into the NVMe memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of an embodiment of a data processing method;

[0054] Figure 2 It is a schematic diagram of the NVMe command format;

[0055] Figure 3 It is a schematic diagram of the NVMe command format;

[0056] Figure 4 It is a schematic diagram of the NVMe command format;

[0057] Figure 5 It is a schematic diagram of the NVMe command format;

[0058] Figure 6 It is a diagram of the mapping relationship between key and value storage locations and metadata storage locations;

[0059] Figure 7 It is a topological diagram of an embodiment of a data processing system hardware topology;

[0060] Figure 8 is a flow chart of an embodiment of a data processing method;

[0061] Fig. 9 It is a command format diagram;

[0062] Fig.10 It is a command format diagram;

[0063] Fig.11 It is a command format diagram;

[0064] Fig.12 It is a flow chart of an embodiment of a method for writing data;

[0065] Fig.13 is a flow chart of an embodiment of a method for reading data;

[0066] Fig.14 It is a flow chart of an embodiment of a method for writing data;

[0067] Fig.15 A logical function diagram of a data processing system embodiment. DETAILED DESCRIPTION

[0068] Non-Volatile Memory Express (NVMe) is a logical device interface that supports access to non-volatile storage media using the PCIe bus. The NVMe interface can be used for flash media storage, such as solid-state drives (SSDs). Devices with NVMe interfaces are called NVMe devices. NVMe memory is a type of NVMe device, which refers to an NVMe device with storage function. The following mainly uses NVMe memory (in the embodiments of the present invention, NVME memory is referred to as memory) as an example for introduction.

[0069] KV storage, also known as K / V storage, is a type of storage technology. In KV storage, key-value / keyvalue pairs are the basic data model. Key-value pairs can include keys and values. After key-value pairs are expanded, they can also include metadata. A key uniquely identifies a value.

[0070] If the host sends KV data to the NVMe storage in blocks for storage, the operations that the host needs to perform include: first, convert the KV data into blocks; then, assign the logical block address (LogicBlock Address, LBA) on the storage to the block; then, generate a write command, which carries the block and the LBA of the block; after the NVMe storage receives the block, it stores the block according to the physical address corresponding to the LBA. If you want to read these KV data, you need to read the block data first, and then convert the block data into KV data. In the process of writing KV data, allocating LBA addresses and converting KV data into block data requires a lot of host time and system resources, increasing system latency. In the process of reading KV data, the host needs to send the location information of the block to be read (such as the starting address + length) to the NVMe storage. After obtaining the block data, converting the block data back into KV data will also consume a lot of host computing resources, increasing system latency.

[0071] This application can be applied to NVMe scenarios and NVMe Over Fabric (NOF) scenarios. In the NVMe scenario, the host and the memory are connected through a bus (such as a PCIe bus). In this scenario, the memory is a component of the host. For example, the host is a server and the memory is an NVMe interface solid-state drive (SSD) in the server. In the NOF scenario, the host and the memory are connected through a fabric (such as Ethernet, FC, etc.). The host includes a memory, a processor, and an interface. The host and the memory can be two independent devices. In an embodiment of the present invention, the host is the initiator of reading and writing data, so it is called an initiator; the memory is the responder of a read request or a write request, also called a target.

[0072] The embodiment of the present invention can extend the existing NVMe protocol and propose a new NVMe command. The new NVMe command extends and optimizes the existing NVMe command (such as the NVMe command defined in the NVMe standard protocol 1.2.1 or the NVMe command defined in the NVMe over Fabric standard protocol 1.0), so that the NVMe device can directly support the KV interface.

[0073] Since NVMe devices can directly support KV interfaces, KV data can be directly transmitted between the storage and the host through the NVMe protocol. The host can directly write KV data into the storage through the NVMe protocol, or read KV directly from the storage through the NVMe protocol. Therefore, the repeated conversion between the KV format and the Block format as in the prior art is avoided between the storage and the host, which reduces the system complexity and improves the system performance. Unless otherwise specified, the NVMe commands in the various embodiments of the present invention refer to extended NVMe commands.

[0074] In an optional implementation, the key is directly carried in the command header of the NVMe command, so the key can be obtained by directly reading the NVMe command header (without parsing the payload of the NVMe command).

[0075] The value can be carried in the NVMe command, and the value can be obtained by reading the NVMe command. In addition, the NVMe command may not directly carry the value, but carry a pointer to the value, and the value pointer directly or indirectly points to the storage space of the value, and the value can be obtained from the storage space of the value through DMA / RDMA technology.

[0076] The following is an exemplary introduction to the terms that may appear in various embodiments of the present invention.

[0077] In the existing NVMe write command, add the following parameters, or define a new NVMe KV write command, which may include the following parameters.

[0078] Number of KV: The number of KVs transmitted in one NVMe command. KV includes key and value corresponding to the key. Or KV includes key and value pointer, value pointer points to value storage space, value and key correspond.

[0079] Value: Data that can be stored in or read from memory, such as a movie.

[0080] Key: uniquely identifies a value. A key can also be called a key code. The combination of a key and its corresponding value can be called a KV or a KV pair.

[0081] KV (key value): A combination of key and value, also called a KV pair. Optionally, KV also includes metadata.

[0082] metadata: used to describe the attributes of a value. For example, if the value is a movie, the metadata may include but is not limited to: the movie title, duration, starring actors, etc.

[0083] Common Header: It is a part of the command header, which refers to the part of the command header that is the same as the existing NVMe command header.

[0084] KV Format ID: Indicates the format of the current NVMe command. In other words, it defines the content of each field in the NVMe command. NVMe commands that transmit KV can have multiple command formats, which are distinguished by KV Format ID. NVMe messages with different command formats can have different fields or different field arrangement orders.

[0085] Key Length: describes the length of the key.

[0086] Value Length: describes the length of the value.

[0087] Metadata Length: describes the length of the Metadata.

[0088] Key Offset: describes the offset position of the key in the NVMe command.

[0089] Value Offset: describes the offset position of value in the NVMe command.

[0090] Metadata Offset: describes the offset position of metadata in NVMe commands.

[0091] DPTR (Data Pointer): Data pointer, pointing to the data to be transmitted.

[0092] MPTR (Metadata Pointer): Metadata pointer, pointing to the metadata to be transmitted.

[0093] PRP (Physical Region Page) Entry: A physical region page instance (or physical region page format entry) that can record a pointer. PRP is one of the two commonly used data transmission protocols in the NVMe protocol and can be used in the NVMe over PCIe architecture. PRP entry can point to a pointer to a physical memory page.

[0094] SGL (Scatter Gather List) Entry: Scatter Gather Table instance (or Scatter Gather Table format entry), which can record pointers. SGL is one of the two commonly used data transmission protocols in the NVMe protocol and can be used in NVMe over PCIe / Fabric architecture.

[0095] See also Figure 1 The present invention provides an embodiment of a data processing method, which can be used between an NVMe storage device and a host.

[0096] 11. The host generates an NVMe write command (hereinafter referred to as a write command) and sends it to the storage. The write command carries the value and the key corresponding to the value. The write command is sent to the NVMe storage through the host's NVMe interface. The key and value can be carried in the payload or in the command header.

[0097] In this step, the key can be carried in the command header or in the payload. In the embodiments of the present invention, unless otherwise specified, the fields carried in the write command / read command can be in the command header or in the payload. For example, the value can be carried in the command header or in the payload.

[0098] Optionally, if the write command carries more than one KV, the write command can also carry number of KVs to describe the number of KVs. After receiving the write command, the storage device can read the KVs according to the number of KVs indicated by number of KVs, and stop reading after reading all KVs. In addition, the end character can be used to indicate the end of the KV, and the reading stops after the end character is read.

[0099] Optionally, the write command further carries a KV Format ID. The NVMe command for transmitting KV can have multiple command formats. In different command formats, the fields can be different and the positions of the fields can also be different. The KV Format ID is used to indicate the command format used by this command.

[0100] The position of the key in the command header of the write command can be unfixed or a pre-set fixed position. If the position of the key in the write command is unfixed, the write command can further carry key position information. The key position information can be a combination of key length and key offset, where key offset describes the starting position of the key in the write command. The key position information can also be a combination of the starting position of the key in the write command and the ending position of the key in the write command. If the starting position of the key in the write command is pre-set, the key position information can be key length, where key length describes the length of the key.

[0101] There are two schemes for presetting the starting position of the key. One is to pre-set the position of the key in the command, for example, starting from the first bit of the command, the key starts from the 20th bit; the other is to pre-set the relative position of the key and other fields, for example, the key field is after the key length field, and the key field is adjacent to the key length field. Similarly, the "pre-set" positions involved in this embodiment and other embodiments can use these two schemes, such as the starting position of the value and metadata to be introduced later in this embodiment.

[0102] The key can be fixed length or variable length. A fixed length key means that the key length is the same in each command. A variable length key means that the key length can be different in different commands. When the key is fixed length, the receiver of the command can ignore the position information (such as key length), or the write command can ignore the position information. After receiving the write command, the memory can directly read the key according to the preset starting position and fixed length.

[0103] The position of value in the write command can be fixed or not. If the position of value in the write command is not fixed, the write command can further carry value position information. The value position information can be valuelength. The starting position of value in the write command is preset, so it does not need to be carried in the write command. Starting from the starting position of value, using value length as the read length, the value can be read from the write command. The value position information can be a combination of value length and value offset. Value offset describes the starting position of value in the write command. The value position information can also be a combination of the starting position of value in the write command and the ending position of value in the write command.

[0104] value can be fixed length or variable length. When value is fixed length, the command receiver can ignore the position information (such as value length), or write the command without carrying the position information. The command receiver can directly read value according to the preset starting position and fixed length.

[0105] Optionally, the write command further carries metadata of the value. The metadata can be carried in the command header or payload of the write command.

[0106] The position of metadata in the write command can be fixed. If the position of metadata in the write command is not fixed, the write command can further carry metadata location information. Metadata location information is similar to value location information, so please refer to the description of value location information, which will not be repeated here. Similarly, metadata can be fixed length or variable length.

[0107] See also Figure 2 It is an optional command format for write commands. The common header is part of the NVMe command header. The command carries 2 KVs, so there are 2 keys, key1 and key2; 2 values, value1 and value2; 2 key lengths, key1 length and key2 length; 2 value lengths, value1 length and value2 length. In addition, the command also carries 2 metadata, metadata1 and metadata2; and the corresponding metadata length1 and metadata length2.

[0108] See also Figure 3 , is another NVMe command format diagram. and Figure 2 Compared with , the value location information (value1offset, value2 offset) and metadata location information (metadata1 offset, metadata1 length, metadata2 offset, metadata2 length) are added, and the number of KV is also added. Figure 3 The command in the example carries two KVs, so the value of number of KV is 2. The location information in the NVMe command describes the location of the field in NVMe. Figure 3The arrow in the figure indicates that the position of the data to be written (eg, the starting position) can be found through the offset. For example, after reading the value1 offset field, the value of the value1 offset field describes the offset position of value1 in the command, so that value1 offset can be read. Figure 3 In the command format, the distribution of fields is determined by the type of field, and fields of the same type are adjacent. For example, value and metadata both belong to the data part, so they are adjacent; value location information and metadata location information both belong to location information, so they are adjacent.

[0109] See also Figure 4 , is another NVMe command format diagram. and Figure 3 In contrast, fields are distributed according to KV affiliation, and fields in the same KV are adjacent. Figure 4 In the command format, the KV1 field comes first, followed by the KV2 field. Figure 4 In the command format, value and value length; metadata and metadata length.

[0110] See also Figure 5 , is another diagram of the NVMe command format. Figure 5 In the same KV, fields are adjacent. In addition, valuelength and metadata length are adjacent, and value and metadata are adjacent.

[0111] 12. The NVMe interface of the storage receives the write command, and the storage obtains the key and value from the write command. If the write command carries metadata, the metadata is also obtained from the write command.

[0112] The memory is composed of a controller and a storage medium. The controller includes a processor and, optionally, a memory. The storage medium is, for example, a flash memory or a disk. The memory can also be a hard disk with management capabilities, called an intelligent hard disk.

[0113] If multiple types of NVMe command formats (such as Figure 2 and Figure 3 If different types of NVMe command formats are supported by the same storage, the storage can determine which format the received command is in through the KV Format ID. Then the key and value contents are read out according to the positions of the key and value fields in this format. The position relationship of data can be different for different NVMe command formats. For example, in some formats, the position of the key is fixed; in some formats, the position of the key is not fixed and is determined by the key position information.

[0114] If the write command carries one KV, then read out this KV. If the write command carries at least two KVs, then read out these at least two KVs. The memory can know the number of KVs through the number of KV field and complete the read operation after reading out the corresponding number of KVs. In addition to using the number of KV to mark when the read operation is completed, you can also add a terminator to the command. After reading the terminator, it means that the KVs in the entire command have been read.

[0115] If the position of the key is fixed, the memory reads the key through the fixed position. If the position of the key is not fixed, the key is read from the write command according to the key position information. For example, the key position information is the key length, and the starting position of the key in the write command is preset. Starting from the starting position of the key, the key length is used as the reading length to continuously read out the data, and the key carried in the write command can be obtained. For example, the key position information is a combination of the key length and the key offset. The key offset is used as the starting position of the key, and the key length and the key length are used as the reading length to continuously read out the data, and the key carried in the write command can be obtained. For example, the key position information is composed of the starting position and the ending position. The key can be obtained by reading the data between the starting position and the ending position.

[0116] The reading methods of value and metadata can refer to the reading methods of key, which will not be repeated here. When reading key / value / metadata, the memory can obtain key length / value length / metadatalength.

[0117] by Figure 2For example, the lengths of the command header, KV format ID, key length, and value length fields are fixed, and the relative positions of the fields are also fixed. Therefore, fields such as keyoffset and value offset are not needed to determine the starting positions of key and value. The order of the memory read and write command fields is: read command header, read KV format ID, read key 1 length, read key1 according to the value recorded in key 1 length, read value1 length, read value1 according to value1 length, read key2 length, read key2 according to key2 length, read value2 length, and read value2 according to value2 length.

[0118] 13. The memory stores the value. Specifically, the memory controller stores the key and value in a non-volatile storage medium. After the memory interface receives the value, it first sends it to the processor. In this step, the processor temporarily stores the value in the memory and then sends it from the memory to the non-volatile storage medium.

[0119] In addition, the memory can also record the mapping relationship between the key and value storage space. The mapping relationship can be stored by the storage medium of the memory; or it can be sent to the host and stored by the host. The following text takes the storage by the memory as an example.

[0120] If the write command also carries metadata, the metadata is also stored, and the mapping relationship between the key and metadata storage space is recorded.

[0121] The value storage space can be described by the starting address of the value storage space and the value length. The metadata storage space can include the starting address of the metadata storage space and the metadata data length. Figure 6 It is a schematic diagram of the mapping relationship, including the mapping of the starting address of the key and value storage space, the mapping of the key and value length, the mapping of the starting address of the key and metadata storage space, and the mapping of the key and metadata length. The key can be used as an index to find the corresponding starting address of the value storage space, the value length, the starting address of the metadata storage space, and the metadata length in the mapping relationship. Figure 6The content described in the diagram is recorded in the KV management unit. If the value and metadata are fixed length, the value length and metadata length are optional.

[0122] The storage space can be a logical location or a physical location, as long as the memory controller can use the storage space to read the value and metadata from the storage medium.

[0123] The above steps 11 to 13 are the process of writing KV, and the following steps 14 to 16 are the process of reading KV. These two processes are independent of each other, and the KV requested by the read command and the KV written by the write command may not be the same.

[0124] 14. The host generates an NVMe read command (hereinafter referred to as the read command) and sends the read command to the memory, wherein the read command carries the key. The key may be carried in the command header of the read command.

[0125] The host that generates the read command and the host that generates the write command can be the same host or different hosts. The command format of the read command refers to the format of the write command in step 11 and Figure 2 The read command includes a command header, a KVformat ID, and a key. Optionally, the read command carries key location information. A read command can carry one key or at least two keys. When carrying at least two keys, the number of KV field can be carried to describe the number of keys carried.

[0126] The read command may not contain the value field and other fields related to the value. In addition, the read command may not contain the metadata field and other fields related to the metadata.

[0127] 15. The storage receives the read command through the NVMe interface and obtains the key from the read command. It searches the value storage space corresponding to the key in the KV management unit stored in the storage. It obtains the value from the value storage space and constructs a response message of the read command and sends it to the host, with the value in the response message.

[0128] Referring to step 12, the method of obtaining the key is slightly different depending on the command format. For example, the key can be obtained from a fixed position in a read command, or from a write command according to the position information carried in the read command.

[0129] Similarly, if metadata corresponding to the key is stored in the memory, the metadata storage space can be queried and the metadata can be obtained in a similar manner.

[0130] It should be noted that if the KV management unit is stored in the host, the key is used to query the value storage space in the host, and the value storage space is sent to the storage device. The storage device obtains the value based on the value storage space and sends it to the host.

[0131] 16. The host receives and stores the value, for example, in a memory (such as a cache) of the host, or in a non-volatile storage medium of the host.

[0132] In the above steps 11-16, both the value and metadata are carried in the NVMe command, and there is no need to convert between KV and block, which has the advantages of being simple and fast. The read command does not need to carry the LBA of the value and / or metadata, so the process of reading the value and / or metadata is also faster.

[0133] See also Figure 7 , is a data processing system hardware topology embodiment of the present invention, a host 71 communicates with a memory 72. The host 71 includes a processor 711, a memory 712, and an interface 713. The memory 72 includes an interface 721, a controller 722, and a storage medium 723. The interface 713 and the interface 721 are connected via a communication link 73, and the communication link 73 is, for example, a PCIe bus, a fiber channel FC, or an Ethernet.

[0134] The operations performed by the host 71 can be performed by the processor 711. For example, the processor 711 can execute steps 11, 14 and 16 by running the program in the memory 712. The memory 712 and the processor 711 are relatively independent or integrated. The memory 72 includes a control area 721 and a storage medium 722. The operations performed by the memory 72 are performed by the controller 721 of the memory. Specifically, the processor 7211 can execute the program in the memory 7212 to execute the operation of the memory, for example, the processor of the memory is used to execute steps 12, 13 and 15. In some cases, such as when the processor is an FPGA, there may be no memory, and the corresponding operations are directly performed by the processor.

[0135] Similarly, the same hardware is used in the subsequent methods, except that the host and the memory are used to perform different operations. For example, steps 21, 24, and 27 are performed by the host 71, and steps 23 and 25 are performed by the memory 72. In steps 22 and 26, part of the operations are performed by the host 71, and the other part of the operations are performed by the memory 72.

[0136] See also Figure 8The present invention also provides an implementation method, in which the write command does not directly carry the value or metadata, but carries a pointer in the NVMe command, and the value and / or metadata to be written can be obtained through the storage space pointed to by the pointer, or a multi-level pointer method is adopted, where one pointer points to another pointer, and the value and / or metadata to be written is obtained from the storage space pointed to by another pointer. This implementation method has no limit on the size of the value and / or metadata to be written.

[0137] The host and storage can be connected through networks such as IP and FC, and can run on the Fabric architecture, also known as the NOF (NVMe Over Fabric) architecture. Under the NOF architecture, the storage can obtain value and / or metadata through remote direct memory access (RDMA). If the storage is inside the host, the storage can also obtain value and / or metadata from the host through direct memory access (DMA). Similarly, the host can also directly obtain value and / or metadata from the storage through RDMA / DMA.

[0138] Step 21: The host constructs a write command, which carries a KV, and the KV includes a value pointer and a key of the value. The write command is sent to the NVMe storage through the NVMe interface of the host. The key and the value belong to the same KV pair.

[0139] The value pointer can be carried in the payload or in the command header. Similarly, other data carried in a write command or a read command can be carried in the command header or in the payload.

[0140] In other embodiments, the write command may also carry a key pointer instead of a key. The scheme of obtaining the key according to the key pointer is the same as the principle of obtaining the value through the value pointer, so it is not described in detail below. Only the write command carrying the key is used as an example for introduction.

[0141] The value pointer directly or indirectly points to the storage space of value (for the convenience of description, it is also referred to as the first storage space below), so the value can be obtained through the value pointer. When the value pointer points to the value storage space (this situation is also regarded as the value pointer pointing to the value), the value can be obtained from the storage space pointed to by the value pointer. In another case, the value pointer points to the first pointer, and the first pointer points to the storage space of the value. The first pointer is found from the value pointer, and the value can be obtained from the storage space pointed to by the first pointer. In the latter case, a larger value can be carried. In other embodiments, there can be more levels of references between pointers, as long as the storage space of the value can be found in the end, for example, the value pointer points to the first pointer, the first pointer points to the A1 pointer, the A1 pointer points to the A2 pointer, ..., the AN-1 pointer points to the AN pointer, and the AN pointer points to the storage space of the value, and N is an integer greater than or equal to 2. The value pointer can be carried in the command header or payload of the write command.

[0142] The first storage space can be described by the first address of the first storage space and the value length, or by the first address and the last address of the first storage space. For example, the value pointer records the first address and the value length of the first storage space; or, the value pointer points to another pointer, and the other pointer records the first address and the value length of the first storage space. The first storage space can be a logical location or a physical location. As long as the memory uses the first storage space, it can read the stored data from the storage medium.

[0143] Optionally, if the write command carries more than one KV, the write command can also carry Number of KV, which is used to describe the number of KVs.

[0144] Optionally, the write command further carries a KV Format ID. The NVMe command for transmitting KV may have multiple command formats. In different command formats, the fields may be different and the positions of the fields may also be different. The KV Format ID indicates the command format used by this command.

[0145] The position of the key in the command header of the write command can be fixed or a preset fixed position. The characteristics of the key can be found in the description of the key in step 11, which will not be repeated here.

[0146] Optionally, the write command further carries a metadata pointer. Similar to the value pointer, the metadata pointer directly or indirectly points to the metadata storage space, so the metadata can be obtained through the metadata pointer. The metadata pointer can be carried in the command header or payload of the write command.

[0147] The metadata pointer directly or indirectly points to the storage space of metadata, and metadata can be obtained through the metadata pointer. When the metadata pointer directly points to metadata (specifically, the storage space of metadata), metadata can be obtained from the storage space pointed to by the metadata pointer; in another case, the metadata pointer points to the second pointer, and the second pointer points to the storage space of metadata. The second pointer is found from the metadata pointer, and metadata can be obtained from the storage space pointed to by the second pointer. In the latter case, longer metadata can be carried. In other embodiments, there can be more levels of references between pointers, as long as the storage space of metadata can be found in the end, for example, the metadata pointer points to the second pointer, the second pointer points to the B1 pointer, the B1 pointer points to the B2 pointer, ..., the BN-1 pointer points to the BN pointer, and the BN pointer points to the storage space of metadata, where N is an integer greater than or equal to 2.

[0148] In addition, metadata can be carried directly in the command instead of being transmitted through pointers. For details, please refer to step 11 and Figure 2 , Figure 3 How to carry metadata in .

[0149] The value pointer can be a data pointer (DPTR). The metadata pointer can be a metadata pointer (PMTR).

[0150] See also Fig. 9 , is a command format diagram. Fig. 9The command carries two KVs, the key in the KV is directly carried in the write command, and the value is carried in the write command in the form of a pointer. They are key1 and DPTR1 (pointing to value1), and key2 and DPTR2 (pointing to value2). In addition, the metadata of these two KVs are also carried in the form of pointers, where MPTR1 points to metadata1 and MPTR2 points to metadata2. In an embodiment of the present invention, each command can carry one KV or more than two groups of KVs.

[0151] See also Fig.10 , is another command format diagram. The pointer (DPTR, MPTR) in the command does not directly point to the value or metadata, but points to the PRP entry or SGL entry. The PRP entry or SGL entry is a node in the linked list. Each PRP / SGL entry points to another address. The address pointed to by the PRP / SGL entry stores the value or metadata. In other words, the write command points to the value or metadata through a secondary pointer. Fig.10 In the scheme, the key is carried in the command, specifically written in the command header.

[0152] See also Fig.11 The command format is, and Fig.10 The difference is that Fig.10 The key is not directly carried in the command, but is transmitted through a pointer. For details on how to carry it, please refer to the introduction of the value pointer. The key pointer can be carried in the command header.

[0153] Specifically, DPTR points to PRP / SGL entry1, and PRP / SGL entry2 and PRP / SGL entry1 belong to the same linked list, so finding PRP / SGL entry1 means finding PRP / SGL entry2. Therefore, it can be considered that DTPR points to PRP / SGL entry1 and PRP / SGL entry2. PRP / SGL entry1 points to value1, and PRP / SGLentry2 points to value2. Similarly, MPTR points to PRP / SGL entry3 and PRP / SGL entry4, and PRP / SGLentry3 points to metadata1, and PRP / SGL entry4 points to metadata2. With DPTR, you can find the storage space of value1 and the storage space of value2; with MPTR, you can find the storage space of metadta1 and the storage space of metadata2. PRP entry and SGL entry are both nodes in the linked list. PRP is applicable to ordinary NVMe (connecting the host and storage with PCIe), and SGL is applicable to NOF (connecting the host and storage with Fabic). RDMA can use PRP, and DMA can use PRP or SGL.

[0154] PRP transmits the location of the data to the NVMe device through a series of pointers to memory pages. After receiving these addresses, the NVMe device can read the data from the host to the NVMe device through DMA. The SGL transmission mechanism is more flexible than PRP. It can specify the transmission length and skip some address spaces during the continuous transmission of addresses. SGL transmits the address of the data to be transmitted to the NVMe device. After receiving these addresses, the NVMe device reads the data from the host to the NVMe device through DMA.

[0155] In addition, compared to Fig. 9 , Fig.10 The number of KV field and the key length field are also added, which have been introduced in steps 11 and 12 and will not be repeated here.

[0156] Step 22, the memory receives a write command through the NVMe interface. The memory obtains the length of value through the value pointer, allocates storage space for value according to the length of value (hereinafter referred to as the second storage space for the convenience of description), and sends the location information of the allocated storage space to the host. The memory sends a transfer request (first transfer request) to the host to obtain value from the first storage space of the host. If the host and the memory are connected by PCIe, the first transfer request may be a DMA transfer request, such as a single DMA transfer request. If the host and the memory are connected by Fabric, the first transfer request may be an RDMA transfer request, such as a single RDMA transfer request. The first transfer request uses the address of the first storage space as the access address and the address of the second storage space as the write address. In the first transfer request, the access address may be: the first address of the first storage space + the length of value, and the write address may be: the first address of the second storage space (or the first address of the second storage space + the length of value).

[0157] The memory sends a DMA / RDMA transfer request to the host. After receiving the transfer request, the host executes DMA / RDMA transfer and sends the value to the memory.

[0158] Pre-allocating the second storage space is optional, and the memory may not allocate the second storage space in advance, directly obtain the value from the host, and then allocate the second storage space for the value. In this case, the first transmission request does not carry the write address.

[0159] If the write command carries a metadata pointer, storage space (the fourth storage space) is also allocated for the metadata. Since the processing method of the metadata pointer is similar to that of the value pointer, only the value pointer will be introduced later.

[0160] Before obtaining value according to the value pointer, a storage space (second storage space) for storing value is allocated in the memory according to the value length, and the second storage space is not less than the value length. The second storage space can be described by the first address + the last address, or by the first address + the value length, or by the first address only. The value length can be determined by the value storage space. For example, in a write command, the first storage space is described by the starting address of the space storing the value + the value length, and the value length can be obtained directly from the write command.

[0161] The memory is composed of a controller and a storage medium. The controller includes a processor, and the storage medium is, for example, a flash memory or a disk.

[0162] The command header, KV format ID, key, and key length fields have been introduced in step 12. How to obtain them and how to use them to obtain corresponding information will not be repeated here.

[0163] Different from the embodiment described in steps 11-16, in this embodiment, the value and / or metadata is not obtained directly from the command, but is obtained by relying on a pointer.

[0164] The value pointer carried in the write command directly points to the first storage space, for example Fig. 9 In the command format, the pointer DPTR in the write command points to the value storage space (the first storage space). Specifically, the pointer DPTR points to a memory address range of the host, which stores the value. Then, according to the storage space described by DPTR, the value and are obtained from the host memory address in DMA or RDMA. In the same way, metadata can be obtained through MPTR.

[0165] If the value pointer in the write command indirectly points to value, you can refer to Fig.10 The format of the write command: the value pointer directly points to another pointer, and the other pointer points to value. The pointer DPTR in the write command points to the PRP / SGL entry, and the PRP / SGL entry points to the host memory address. The host memory address is the storage space of value. The memory first finds the PRP / SGL entry through DPTR, and then finds the value storage space located in the host memory through the PRP / SGL entry, and obtains the value from the host memory in DMA or RDMA.

[0166] If the metadata pointer is carried in the write command, the method of obtaining the metadata is the same as the method of obtaining the value. If the metadata is directly carried in the write command, the metadata can be directly obtained from the write command with reference to step 12.

[0167] The memory sends a first transfer request to the host to obtain value from the first storage space of the host. If the host and the memory are connected by PCIe, the first transfer request can be a DMA transfer request; if the host and the memory are connected by Fabric, the first transfer request can be an RDMA transfer request. Specifically, it can be divided into two steps: (1) The host sends a transfer request to the host, and the transfer request carries the first storage space as the access address and the second storage space as the write address. Specifically, the first storage space carried in the transfer request can be described by the first address of the first storage space and the length of the value; the second storage space is described by the first address of the second storage space. (2) After receiving the transfer request, the host reads data from the first storage space and sends it to the second storage space of the memory for storage.

[0168] In this step, the key, value, and metadata can be transferred through a single DMA / RDMA transmission. Alternatively, the key, value, and metadata can be transferred separately.

[0169] In step 23, the memory stores the value. Specifically, the controller of the memory may store the key and the value in the storage medium of the memory. If metadata is obtained in step 22, the metadata is also stored.

[0170] For example, starting from the first address of the second storage space, the value is continuously stored in the storage medium of the memory.

[0171] In addition, the mapping relationship between key and value storage space can also be recorded, see Figure 6 , Figure 6 The value storage space is described by the first address + value length. The mapping relationship is recorded in the KV management unit. The value length and metadata length are optional, because both the value and metadata can be fixed length.

[0172] In the above steps 21-23, value is transmitted in the form of a pointer. In other embodiments, key can also be transmitted in the form of a pointer (direct pointer or indirect pointer), which can be specifically referred to in the transmission method of value, and will not be described in detail here.

[0173] The above steps 21-23 are the process of writing commands, and steps 24-26 are the execution process of read commands. These two processes are independent of each other. The KV requested by the read command and the KV written by the write command may not be the same. The command formats of the read command and the write command are similar, except that the read command does not carry a value pointer and a metadata pointer.

[0174] 24. The host generates a read command and sends it to the memory, wherein the read command carries the key. The key may be carried in a command header of the read command.

[0175] The host that generates the read command and the host that generates the write command can be the same host or different hosts. The format of the read command refers to the format of the write command in step 11 and Figure 2 , the read command includes a command header, a KVformat ID, and a key. Optionally, the read command carries a key or a key pointer. A read command can carry one key or at least two keys. When carrying at least two keys, the number of KV field can be carried to describe the number of keys carried.

[0176] The read command requests to read the value and / or metadata.

[0177] Compared with a write command, a read command may not have a value field and value location information, and may not have a metadata field and metadata location information.

[0178] 25. The memory receives the read command through the NVMe interface and obtains the key from the read command. Using the obtained key, the storage space of the value is searched from the mapping relationship stored in the memory and sent to the host. The storage space of the value in the memory is the second storage space.

[0179] If the key is directly carried in the read command, refer to step 12. The method of obtaining the key from the read command varies slightly depending on the command format. For example, the key can be obtained from a fixed position in the read command; if the position of the key in the read command is not fixed, the key can be obtained from the read command according to the position information carried in the read command.

[0180] Optionally, if metadata corresponding to the key is stored in the memory, the storage space of the metadata can also be obtained and sent to the host.

[0181] 26. The host allocates storage space (named as the third storage space) to value according to the length of value. The host constructs a transfer request (the second transfer request), sends the transfer request to the memory, and obtains value from the second storage space of the memory.

[0182] If the host and the memory are connected by PCIe, the second transfer request can be a DMA transfer request, such as a single DMA transfer request; if the host and the memory are connected by Fabric, the second transfer request can be an RDMA transfer request, such as a single RDMA transfer request. In the transfer request, the access address can be: the first address of the second storage space + the value length, and the write address can be the first address of the third storage space (or the first address of the third storage space + the value length). It can be seen from the above steps that the second storage space is used to store the value, so the value length is the same as the size of the second storage space. That is to say, the size of the second storage space is equal to the size of the third storage space.

[0183] The value length can be determined by the value storage space. For example, in the read command, the second storage space is described by the starting address of the space storing the value + the value length, and the value length can be directly obtained from the write command.

[0184] The host sends a DMA / RDMA transfer request to the memory. After receiving the transfer request, the memory executes DMA / RDMA transfer and sends the value to the host.

[0185] Pre-allocating the third storage space is optional. The host may also obtain the value directly from the memory without allocating the third storage space in advance, and then allocate the third storage space for the value. In this case, the transfer request does not carry the write address.

[0186] Similarly, the host can allocate storage space to metadata according to the metadata length and obtain metadata from the storage.

[0187] 27. The host stores the received value into the third storage space. For example, starting from the first address of the third storage space, the value is continuously stored into the host. The third storage space is, for example, in the memory of the host.

[0188] The metadata is stored in the host in a similar manner as the value is stored in the host.

[0189] It should be noted that, in other embodiments, DMA or RDMA may not be used, and steps 26 and 27 may be modified as follows: the host allocates storage space to the value, the memory reads the value from the second storage space, and writes the read value into the third storage space.

[0190] In addition, another optional scheme of step 26 is: the NVMe read command sent by the host received by the NVMe memory may further carry the free space information of the host, and the free space information is used to describe the size of the continuous free storage space in the host. After the NVMe memory receives the NVMe read command from the host, it further includes: the NVMe memory determines whether the free storage space of the host is greater than or equal to the second storage space, if so, executes the step of sending the location information of the second storage space to the host, if not, ends the step. In this scheme, the continuous free storage space in the host can be called the third storage space.

[0191] See also Fig.12 , taking the value pointer and metadata pointer carried in the same command as an example, steps 21 to 23 are introduced.

[0192] Step 31, the host constructs a write command. The write command carries a key. In addition, the write command carries a value source address list and a metadata source address list. The source address list has a pointer function, and the value and metadata source address lists point to the value storage space and metadata storage space respectively.

[0193] Step 32, the host sends a write command to the NVMe device.

[0194] Step 33: After receiving the write command, the NVMe device parses the value source address linked list and the metadata source address linked list from the write command. The linked list is, for example, a PRP entry or an SGL entry.

[0195] Step 34, the NVMe device parses the value length from the value source address linked list, and parses the metadata length from the metadata source address linked list. The source address linked list describes the storage space of the data to be read, so the value length and metadata length can be parsed from it.

[0196] Step 35, the NVMe device sends a DMA or RDMA request to the host. The request carries a value source address list, a metadata source address list, a value destination address list, and a metadata destination address list. The value destination address list describes the storage space reserved by the NVMe device for storing values, and the metadata destination address list describes the storage space reserved by the NVMe device for storing metadata.

[0197] Step 36: After receiving the DMA / RDMA request, the host sends the key to the NVMe device according to the source address and destination address of step 35.

[0198] In step 37, the NVMe device stores the value and metadata in the medium.

[0199] Step 38, the NVMe device records the storage space of the value and the storage space of the metadata. The storage space and the key are mapped so that the key can be used to find the value storage space and the metadata storage space later. The mapping relationship can be stored in the KV management unit, and the KV management unit can also record the value length and metadata length.

[0200] In the above steps, the key is directly carried in the command. In other implementations, the command may not carry the key but carry the key pointer. After receiving the command, the memory obtains the key according to the key pointer. The specific steps for obtaining the key can refer to the process of obtaining the value in step 22. Since the principle is similar, it will not be repeated here.

[0201] See also Fig.13 , taking the value pointer and metadata pointer carried in the same command as an example, the specific description of steps 24-27 is given.

[0202] Step 41, the host constructs a read command, and the read command carries a key.

[0203] Step 42, the host sends a read command carrying the key to the NVMe device.

[0204] Step 43, the NVMe device searches the KV management unit for the value length and metadata length corresponding to the key, wherein the KV management unit records the value storage space and the metadata storage space.

[0205] Take the value storage space as an example. The value is stored in the NVMe device. The value storage space is described by the value first address + the last address, or by the first address + the value length. Therefore, after obtaining the value storage space, the value length can be obtained. Similarly, the metadata length can be obtained.

[0206] Step 44, the NVMe device sends a response message to the host, and the response message carries the value length and metadata length.

[0207] Step 45, the host allocates storage space to the value and metadata according to the value length and metadata length for future storage of the value and metadata.

[0208] Step 46: The host sends a DMA / RDMA request to the NVMe device, and the request carries the source address and destination address of this transmission. The source address is the value storage space and metadata storage space in NVMe, and the destination address is the storage space reserved by the host for value and the storage space reserved for metadata.

[0209] Step 47, after receiving the DMA / RDMA request from the host, the NVMe device sends the value and metadata according to the DMA / RDMA protocol.

[0210] Step 48, the host receives the value and metadata.

[0211] In step 49, the host stores the received value and metadata, for example, in the host memory.

[0212] DMA or RDMA is usually used to read data from a continuous address space. When the storage space for storing value and the storage space for storing metadata are not continuous, they can be processed as two DMA / RDM transfers.

[0213] See also Fig.14 The present invention further provides an embodiment based on steps 21-23, based on Fig.12 The improvement is that the key, value and metadata of the same command are obtained by three DMA / RDMA transmissions.

[0214] Optionally, if any of the transmissions fails, it means that the KV transmission is unsuccessful, so the successfully transmitted data can be deleted and the error code uploaded. Subsequent data transmission is canceled, and the storage space previously allocated for the KV (key, value, and metadata) can be released.

[0215] Step 51, construct an NVMe write command.

[0216] The write command carries the key. In addition, the write command carries the value source address list and the metadata source address list. The source address list has the function of a pointer, and the value and metadata source address lists point to the value storage space and metadata storage space respectively.

[0217] Step 52, the host sends a write command to the NVMe device.

[0218] Step 53: After receiving the write command, the NVMe device parses the value source address linked list and the metadata source address linked list from the write command. The linked list is, for example, a PRP entry or an SGL entry.

[0219] Step 54, the NVMe device parses the key length from the key source address list, parses the value length from the value source address list, and parses the metadata length from the metadata source address list. Storage space is reserved for key, value, and metadata. The source address list describes the storage space for the data to be read, so the value length and metadata length can be parsed from it.

[0220] Step 55, the NVMe device sends a DMA / RDMA request to the host. The request carries a key source address list and a key destination address list. The key destination address list describes the storage space reserved by the NVMe device for storing keys.

[0221] Step 56, after receiving the DMA / RDMA request, the host sends the key to the NVMe device according to the source address and destination address of step 55.

[0222] Step 57, if the key transmission fails, the NVMe device cancels the subsequent value and metadata transmission. Optionally, the storage space reserved for the key, value, and metadata is released. An error code is reported to the host. If the key transmission is successful, proceed to the subsequent step 58.

[0223] Step 58: The NVMe device sends a DMA / RDMA request to the host. The request carries a value source address list and a value destination address list. The value destination address list describes the storage space reserved by the NVMe device for storing the value.

[0224] Step 59: After receiving the DMA / RDMA request, the host sends the value to the NVMe device according to the source address and destination address of step 58.

[0225] Step 60, if the value transmission fails, the NVMe device cancels the subsequent metadata transmission. Optionally, release the storage space reserved for the value and metadata. Optionally, delete the key that has been transmitted. Report an error code to the host. If the value transmission is successful, continue to the subsequent step 61.

[0226] Step 61, the NVMe device sends a DMA / RDMA request to the host. The request carries a metadata source address list and a key destination address list. The metadata destination address list describes the storage space reserved by the NVMe device for storing keys.

[0227] Step 62, after receiving the DMA / RDMA request, the host sends metadata to the NVMe device according to the source address and destination address of step 61.

[0228] Step 63, if the metadata transmission fails, optionally, release the storage space reserved for metadata. Optionally, delete the key and value that have been transmitted. Report an error code to the host. If the value transmission is successful, continue to the subsequent step 64.

[0229] Step 64, store the value and metadata to the memory. The value and metadata obtained in steps 60 and 63 are in the memory. This step stores the value and metadata to a non-volatile medium, such as a hard disk of an NVMe device.

[0230] Step 65, record the storage space of value and the storage space of metadata.

[0231] from Fig.14 It can be seen from steps 55 to 63 that the key, value and metadata are obtained through different DMA transmissions respectively. If any data transmission fails, the transmission of the remaining data is canceled, and the data that has been successfully transmitted can be deleted. Fig.14 The remaining steps and Fig.12 Similar to this, it has been introduced in the previous article and will not be repeated here.

[0232] Similarly, except that value and metadata can be transmitted separately. Value (or metadata) can also be transmitted multiple times, for example, a value can be divided into two or more DMA / RDMA transmissions. If any transmission fails, the entire KV transmission is determined to have failed, and the storage space reserved for key, value, and metadata is released. When the NVMe memory and the host are connected with a PCIe bus, the memory sends at least two DMA transfer requests to the host to obtain the value, and each DMA transfer obtains a portion of the value. When the NVMe memory and the host are connected with Frabic, the memory sends at least two RDMA transfer requests to the host to obtain the value, and each RDMA transfer obtains a portion of the value.

[0233] The scheme of transmitting value (or metadata) in multiple times can be combined with the scheme of transmitting value and metadata separately described in step 55 to step 63. In this case, if any transmission fails (or the transmission request fails to execute), the entire KV transmission is considered to have failed, and the storage space reserved for key, value, and metadata is released.

[0234] Fig.14 The process described in the figure is how to handle write commands. The initiator of the DMA / RDMA transfer request is the NVMe device. Similarly, in the processing flow of read commands, a similar solution can be adopted. The key, value, and metadata are obtained through different DMA transfers. The difference is that the initiator of the transfer request is the host, and the NVMe device transmits the key, value, and metadata through DMA / RDMA.

[0235] See also Fig.15 The present invention provides a data processing system, including a host device 80 and a storage device 90, wherein the host device 80 and the storage device 90 are connected via PCIe or Fabric. The host device 80 can read or write KV data into the storage device 90.

[0236] The host device 80 may be a physical device or a logical device, including a transceiver module 801, a management module 802 and a cache module 803. The storage device 90 may be a physical device or a logical device, including an interface module 901, a processing module 902 and a storage module 903. The transceiver module 801 communicates with the interface module 901. The host device 80 has the aforementioned host function, and the storage device 90 has the aforementioned memory function.

[0237] The functions of the host device 80 and the storage device 90 are briefly introduced below. It should be noted that since the functions of these two devices (and corresponding modules) have been described in detail in the method flow, only a brief introduction is given here.

[0238] The storage device 90 includes: an interface module 901, which is used to receive an NVMe write command sent by a host, wherein the NVMe write command carries a key, and the NVMe write command carries a value pointer, wherein the value pointer points to a first storage space in the host, wherein the first storage space is used to store a value, and the key and the value belong to the same KV pair; a processing module 902, which is used to obtain the key from the NVMe write command, obtain the value length according to the value pointer, and allocate a second storage space for the value according to the value length; a storage module 903, which is used to send a first transmission request to the host, obtain the value from the host, and save the value in the second storage space. The second storage space can be provided by a storage medium in the NVMe memory, and the storage medium is connected to a processor of the NVMe memory. Based on this solution, in the process of transferring KV data from the host to the NVMe memory, it is not necessary to convert the KV data into a block form, thereby improving the storage efficiency of the KV data.

[0239] Or the storage device 90 includes: an interface module 901, used to receive an NVMe write command, the header of the NVMe write command carries a key key, the NVMe command also carries a value value, the key and the value correspond to each other, and the key and the value belong to the same KV pair; a processing module 902, used to obtain the key and the value from the NVMe write command; a storage module 903, used to save the value in the storage medium of the NVMe storage.

[0240] In one embodiment, the host device 80 includes: a transceiver module 801, which is used to send an NVMe read command to the storage device 90, and the NMVe command carries a key; the transceiver module 801 is also used to receive a response message of the NVMe read command from the NVMe memory, and the response message carries the location information of the second storage space; a management module 802 is used to reserve a third storage space for the value according to the value length in the value location information; the transceiver module 801 is also used to send a transfer request to the NVMe memory, and the access address of the transfer request is the second storage space, and obtain the value from the NVMe memory; a cache module 803 saves the value in the third storage space of the host device 80. Based on this module structure, the host device can read the value and / or metadata from the storage device.

[0241] The units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0242] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions based on the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: The fast non-volatile storage NVMe memory receives an NVMe command sent by a host, wherein the NVMe command carries a field in a key value KV format, wherein the field in the KV format describes a structure of a field in the NVMe command; The NVMe storage device obtains K and V according to the fields of the KV format, wherein the K and the V are a KV pair, and the KV pair includes a key and a value; The NVMe memory completes processing of the NVMe command according to the obtained K and the V; The NVMe command carries multiple KVs; The NVMe command carries a field for the number of KVs, and the field for the number of KVs is used to describe the number of KVs in the command. The NVMe memory obtains the same number of KVs as indicated by the field for the number of KVs from the command.

2. The method according to claim 1, characterized in that: The KV format field includes the K and a pointer for indicating the location of the V, where the location of the V is a storage location in the host.

3. The method according to claim 1, characterized in that: The KV format field includes the K and the V.

4. The method according to any one of claims 1 to 3, characterized in that: The NVMe command is a write command.

5. The method according to claim 4, characterized in that: The NVMe storage device obtains keys and values ​​according to the fields in the KV format, including: The NVMe storage device obtains the K according to the field in the KV format to determine the storage location and size of the matched V in the host; Then the NVMe memory completes the processing of the command according to the K and the V, including: The NVMe memory divides a first storage space in the NVMe memory according to the size of the value; The NVMe storage obtains the value from the host; The NVMe memory stores the value in the first storage space.

6. The method according to claim 5, characterized in that: The method further comprises: The correspondence between the K and the V is stored in a KV mapping relationship, wherein the correspondence between the K and the V includes a matching relationship between the K and the V, a storage location of the first storage space, and metadata, where the metadata is used to describe the attributes of the V.

7. The method according to claim 4, characterized in that: The NVMe storage completes the processing of the command according to the key and value, including: The NVMe storage parses the NVMe command and obtains the key and value; The NVMe memory divides a first storage space in the NVMe memory according to the size of the value; The NVMe memory stores the value in the first storage space.

8. The method according to any one of claims 1 to 3, characterized in that: The NVMe command is a read command.

9. The method according to claim 8, characterized in that: The NVMe command includes the key, and the NVMe storage obtains the key and the value according to the field in the KV format, including: The NVMe memory queries the position of the value corresponding to the key in the second storage space according to the KV mapping relationship, wherein the second storage space is the storage space of the NVMe memory, and the KV mapping relationship is stored in the NVMe memory; The NVMe storage completes the processing of the command according to the key and value, including: The NVMe storage device sends the location of the second storage space to the host; The NVMe memory receives a transfer request sent by the host, where the transfer request is used to request data stored in the second storage space.

10. A fast non-volatile storage NVMe memory, characterized in that: The NVMe memory is used to execute the operating steps of the method performed by the NVMe memory in any one of claims 1 to 9 of the above method.

11. A fast non-volatile storage NVMe storage system, the system comprising a host and an NVMe memory, the host and the NVMe memory are respectively used to implement the operation steps performed by the corresponding subject in any one of the claims 1 to 9 above.

Images