Modifying NVMe PRP list pointers and data pointers to facilitate routing PCIe memory requests

Abstract

Embodiments of the present disclosure relate to modifying an NVMe PRP list pointer and a data pointer to facilitate routing PCIe memory requests. A first command is received from a virtual or physical host associated with a storage system including two or more hosts. The first command includes one or more physical request page (PRP) entries associated with a non-volatile memory high speed (NVMe) standard. The one or more PRP entries are modified with an indication of the virtual or physical host. The second command is sent with the modified one or more PRP entries to a solid state drive (SSD). A memory request is received from the SSD, wherein the memory request includes the modified one or more PRP entries. A memory request is routed to the virtual or physical host based on an indication of the virtual or physical host in the modified one or more PRP entries.

Description

[0001] This application is a divisional application of an invention patent application with an application date of August 8, 2019 (the earliest priority date is August 8, 2018), a Chinese national application number of 201910730943.2, and a name of “Modifying NVMePRP list pointers and data pointers to facilitate routing PCIe memory requests”. Technical Field

[0002] The present disclosure relates to providing storage services to two or more hosts, which may be virtual hosts and / or physical hosts, namely, to modifying physical region page (PRP) data pointers and list pointers associated with the Non-Volatile Memory Express (NVMe) standard to facilitate routing Peripheral Component Interconnect Express (PCIe) memory requests to one of the two or more hosts. Background Art

[0003] PCI Express (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard, which is officially abbreviated as PCIe or PCI Express. PCI Express operates as a motherboard-level interconnect between a host and peripheral components, such as an attached solid-state drive (SSD). Non-Volatile Memory Express (also known as NVMe or NVM Express) is used to manage data stored on an SSD by providing a command set for SSD access. The host sends NVMe commands to the SSD to store and retrieve data from the SSD. These commands utilize physical region page (PRP) entries to represent buffer locations in the host memory. PRP entries take the form of a data pointer to the data or a list pointer to a list of data pointers (similar to a linked list). The SSD can request data movement associated with the PRP entry when executing a command, such as a PCIe memory read request. When a single host is associated with the SSD, the routing of PCIe memory read requests is clear. PCIe memory read requests are sent to a single host. When two or more hosts (such as physical hosts and / or virtual hosts) are associated with the SSD, routing PCIe memory read requests becomes more complicated. The PRP entry does not provide an indication of which of the two or more hosts is associated with the PRP entry, which would otherwise facilitate routing to that host. Summary of the invention

[0004] The present disclosure relates to a non-volatile memory express (NVMe) switch that facilitates NVMe-based access to multiple solid-state drives (SSDs) by one of two or more hosts, where the host can be a physical host and / or a virtual host. The NVMe switch modifies a physical region page (PRP) entry (such as a data pointer or a list pointer) of an NVMe command with a host ID or a virtual function (VF) ID and a tag, where the host ID or the virtual function (VF) ID is associated with the host that sends the NVMe command to the NVMe switch, and the tag indicates whether the PRP entry is a data pointer or a list pointer. The host ID or the VF ID facilitates routing a peripheral component interconnect express (PCIe) memory request with a modified PRP data pointer or list pointer to one of the two or more hosts associated with the NVMe command to execute the PCIe memory request.

[0005] According to one aspect of the described systems and techniques, a method includes: receiving a first command from a host associated with a storage system including two or more hosts, wherein the first command includes one or more physical request page (PRP) entries associated with a non-volatile memory express (NVMe) standard; modifying the one or more PRP entries with an indication to the host; sending a second command to a solid-state drive (SSD), the second command having the modified one or more PRP entries; receiving a memory request from the SSD, wherein the memory request includes the modified one or more PRP entries; and routing the memory request to the host based on the indication to the host in the modified one or more PRP entries.

[0006] According to another aspect of the described systems and techniques, a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to at least: receive a first command from a host associated with a storage system comprising two or more hosts, wherein the first command includes one or more physical request page (PRP) entries associated with a non-volatile memory express (NVMe) standard; modify the one or more PRP entries with an indication to the host; send a second command to a solid-state drive (SSD), the second command having the modified one or more PRP entries; receive a memory request from the SSD, wherein the memory request includes the modified one or more PRP entries; and route the memory request to the host based on the indication to the host in the modified one or more PRP entries.

[0007] According to yet another aspect of the described systems and techniques, a non-volatile memory express (NVMe) switch includes a command processor and a data processor, the command processor including instructions stored in a memory of the NVMe switch, which instructions, when executed by one or more processors of the NVMe switch, cause the NVMe switch to at least perform: receiving, by the command processor, a first command from a host associated with a storage system including two or more hosts, wherein the first command includes one or more physical request page (PRP) entries associated with a non-volatile memory express (NVMe) standard; modifying, by the command processor, the one or more PRP entries with an indication to the host; and sending, by the command processor, a second command to a solid-state drive (SSD), the second command having the modified one or more PRP entries; and the data processor including instructions stored in the memory of the NVMe switch, and which instructions, when executed by one or more processors of the NVMe switch, cause the NVMe switch to at least perform: receiving, by the command processor, a memory request from the SSD, wherein the memory request includes the modified one or more PRP entries; and routing the memory request to the host based on the indication to the host in the modified one or more PRP entries.

[0008] In this regard, the NVMe switch allows routing of PCIe memory requests associated with NVMe commands generated by a host when the storage system includes two or more hosts. The PCIe memory requests are routed to the host. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1A and Figure 1B An example storage system is illustrated that facilitates handling of physical region pages (PRPs) when the storage system includes two or more hosts.

[0010] Figure 2 is an example flow diagram of functionality associated with processing a PRP entry associated with a Non-Volatile Memory Express (NVMe) command received from one of two or more hosts.

[0011] Figure 3 is an example flow diagram of functionality associated with handling PCIe memory requests from an SSD.

[0012] Figure 4 Illustrated is an example process for modifying a PRP entry associated with an NVMe command, and retrieving and modifying an associated data pointer and / or list pointer associated with a list of pointers retrieved from a host based on the PRP entry.

[0013] Figure 5is a simplified block diagram of an NVMe switch.

[0014] The drawings are for the purpose of illustrating example embodiments, but it is to be understood that the embodiments are not limited to the arrangements and instrumentality shown in the drawings. DETAILED DESCRIPTION

[0015] The present disclosure relates to a non-volatile memory express (NVMe) switch that facilitates NVMe-based access to multiple solid-state drives (SSDs) by one of two or more hosts. The host mentioned herein may be a physical host or a virtual host such as a virtual machine (VM). The NVMe switch modifies a physical region page (PRP) entry (such as a data pointer or a list pointer) of an NVMe command using a host ID or a virtual function (VF) ID and a tag, the host ID or the virtual function (VF) ID being associated with the host sending the NVMe command, the tag indicating whether the PRP entry is a data pointer or a list pointer. The host ID or the VF ID facilitates routing a peripheral component interconnect express (PCIe) memory request with a modified PRP data pointer or a list pointer to one of two or more hosts associated with the NVMe command. The principles described herein may be applied to control the performance of other types of storage devices such as hard disk drives (HDDs) or hybrid SSD / HDD drives.

[0016] Figure 1A and Figure 1B An example storage system is illustrated to facilitate the processing of physical region pages (PRPs) when the storage system includes two or more hosts. For ease of illustration, the example storage system is cross- Figure 1A and Figure 1B, wherein reference numeral A interconnects the structure associated with the two figures. The example storage system 100 may include two or more hosts shown as host 102 and host 104, one or more solid-state drives (SSDs) shown as SSDs 106-112, and NVMe switches 116. Two or more hosts 102-104 may be used as servers that provide information resources, services, and / or applications to users or other nodes on a network. In some examples, the host may be an actual physical host. In other examples, the host may be a virtual machine (VM) or a virtual host. VM or virtual host may be implemented on an actual physical host using PCIe single root input / output virtualization (SR-IOV) and associated with a virtual function (VF). SSD is a storage mechanism based on an integrated circuit (IC) that can store data persistently to enable the host to provide the desired functionality. Compared to a hard drive, SSD106-112 can provide higher bandwidth and lower latency. NVMe switches 116 may be positioned between hosts 102-104 and SSDs 106-112. The NVMe switch 116 controls the PCIe-based point-to-point switch connections between the hosts 102-104 and the SSDs 106-112 so that each SSD 106-112 has its own dedicated connection to the host 102-104.

[0017] NVMe switch 116 includes PCIe endpoint cluster 118, command acquisition (fetch) engine 120, system fabric (fabric) 122, command processor 124, data processor 126 and PCIe root complex (RC) cluster 128. PCIe endpoint cluster 118 can be coupled to two or more hosts 102-104. NVMe is used to manage data stored on SSD 106-112 by providing a command set for SSD access. In an example, two or more hosts 102-104 can implement queue pairs according to NVMe to facilitate SSD access. The queue pair consists of a submission queue and a completion queue. For example, the host can generate NVMe commands to obtain data stored in the SSD or store data in the host's memory to the SSD. When the SSD executes the command, the completion of the execution of the notification command can be stored in the completion queue. Command 130 is an example of a command placed on the submission queue by the host 102, and command 132 is an example of a command placed on the submission queue by the host 104. Each command may be associated with an example command type shown by an OPCODE, an example data operation size shown as "# of Logical Block Addresses (LBAs)", and include a Physical Region Page (PRP) entry with a PRP address that serves as a pointer to identify the location of the data or list in the host's memory domain. NVMe-compliant commands may utilize PRP entries to identify data associated with get and store operations.

[0018] Although only one PRP is shown in commands 130, 132, two PRP entries are typically associated with the PRP entry. The PRP entries may be identified as PRP1 and PRP2. The PRP1 entry may be a PRP data pointer that points to data stored in the host, which in some examples may be 4K data. The data pointer may have a PCIe address of a host memory in which the data is stored. The PRP2 entry may also have a PRP data pointer or a PRP list pointer. If the PRP2 entry is a data pointer, the PRP2 entry points to data stored in the host, which may be 4K data, so that the PRP1 entry and the PRP2 entry allow 8K operations to be performed. If the PRP2 entry is a list pointer, the PRP2 entry points to a list of pointers stored in the host. The list pointer may have a PCIe address of a host memory in which data pointers 2 to n or a combination of data pointers and list pointers are stored.

[0019] The PCIe connection between two or more hosts and the NVMe switch 116 may include 2, 4, 8, or 16 data transmission channels. Each transmission channel may include two pairs of wires, one pair for transmission and one pair for reception. Two or more hosts may be coupled to the NVMe switch 116 via a PCIe connection so that each of the two or more hosts can be uniquely addressable by a PCIe address. The two or more hosts may be actual physical hosts or include VMs. The command acquisition engine 120 may acquire commands from two or more hosts, and then forward the commands to the command processor 124 through the internal nodes of the NVMe switch 116, via the system structure 122, or the cross switch. For example, the command acquisition engine 120 acquires commands from the NVMe submission queue in the host memory to the local memory of the NVMe switch 116 for processing. The command acquisition engine 120 facilitates the delivery of queue entries in the host memory queue submitted by the host CPU to the local memory queue for processing by the local CPU. The command processor 124 determines to which SSD each command will be forwarded, and then the PCIe RC cluster 128 provides the command to the determined SSD. For example, commands 130 associated with host 102 may be sent to SSD 106 , while commands 132 associated with host 104 may be sent to SSD 112 .

[0020] Each SSD receiving the command may generate a PCIe memory request based on the received command. The NVMe switch 116 may receive the request to route it to the host that generates the command associated with the request. The PCIe memory request may include a PRP entry for the command. The PRP entry may include a 64-bit PCIe address to identify the location of the data or list in the memory domain of the host, but it does not typically indicate the host associated with the PRP entry. An indication of the host that generated the PRP entry is required because the NVMe switch 116 needs to route the PCIe memory request with the PRP entry to the host that generated the PRP entry to execute the PCIe memory request.

[0021] In an example, the command processor 124 embeds an indication of the host before sending the command from the host to the SSD to facilitate the routing. The indication is embedded in the PRP address associated with the command, which is sent from the command processor 124 to the SSD via the PCIe RC cluster 128. For example, the command 134 associated with the host 102 and the command 136 associated with the host 104 sent to the SSDs 106, 112 may have different PRP addresses (shown as "0A" and "0B" in the higher bits, or specifically in the upper 2 octets) to indicate that each command is associated with a different host when received by the corresponding SSD. When the SSD sends a PCIe memory request with a PRP entry to the NVMe switch 116 to retrieve data from the host associated with the PRP entry, the SSD includes an indication of the host in the PCIe memory request sent to the NVMe switch 116, so that the NVMe switch 116 has a host ID (i.e., host identifier) ​​for a physical host or a VF ID (i.e., virtual function identifier) ​​for a virtual host or VM. For example, the PCIe memory read request 138 generated by the SSD 106 and the PCIe memory read request 140 generated by the SSD 112 have a PRP entry with an indication of the host embedded by the command processor. The data processor 126 sends the PCIe memory request to the appropriate host via the PCIe EP cluster 118 to perform the operation. For example, the PCIe memory request can be routed to the host based on the host ID and / or VF ID in the PRP entry, and as shown, the PCIe memory read request 138 is routed to the host 102 and the PCIe memory read request 140 is routed to the host 104. The embedded host identification allows the data processor to know where to route the PCIe memory request in order to reach a host that can execute the PCIe memory request. In addition, the data processor 126 can tag the list of pointers obtained from the PCIe memory request with information associated with whether the pointer is a data pointer or a list pointer and the host ID and / or VF ID. In some examples, the host ID or VF ID can also be considered a tag.

[0022] Example Operation

[0023] Figure 2is an example flow chart 200 of functionality associated with processing a PRP entry associated with an NVMe command received from one of two or more hosts. The functionality facilitates subsequent routing of a PCIe memory request to the host that sent the NVMe command, wherein the PCIe memory request is received from an SSD to retrieve a list of data or pointers associated with the PRP entry in the NVMe command. The functionality may be performed via a command processor in firmware, software, hardware, or a combination of hardware, software, and / or firmware.

[0024] At 202, a storage access command may be received from a host. The storage access command may be an NVMe command for reading data from an SSD or writing data to an SSD. The NVMe command may include two PRP entries referred to as PRP1 and PRP2. PRP1 may be a data pointer, and PRP2 may be a data pointer or a list pointer. The pointer may be an address to a host memory having a list of data and / or data pointers.

[0025] At 204, the address of PRP1 is modified to include an indication associated with a data pointer and an indication of a host associated with PRP1. A tag may be inserted into PRP1 to indicate that it is associated with a data pointer. For example, the tag may be a unique set of bits indicating that PRP1 is associated with a data pointer. The NVMe command may be associated with metadata indicating which host sends the command. The metadata may be indicated by routing information for sending the NVMe command from the host to the NVMe switch. PRP1 may be modified to include a host ID, which is used to indicate the host that sends the NVMe command. In some examples, PRP1 may be modified to include a virtual function (VF) ID. The VF ID may indicate a VM or a virtual host to uniquely identify the VM or virtual host. Data tags, VF IDs, and / or host IDs may be inserted in many ways. For example, the higher address bits of the address associated with PRP1 may not be used or reserved, and are replaced by data tags and hosts associated with PRP1 (e.g., VF IDs and / or host IDs). The NVMe switch may store a copy of the replaced bits.

[0026] At 206, it is determined whether PRP2 is valid. The validity of the PRP2 entry can be based on the product of the input / output (I / O) size associated with the command, indicated in some examples by a "# of LBA" entry in the NVMe command, and the size of the LBA can be a fixed value established during host configuration. The I / O size indicates the amount of data associated with the data pointer referenced by PRP1 and PRP2. For example, if the I / O is 4K, PRP1 may be sufficient to point to the data associated with the NVMe command. PRP2 does not point to any additional data pointers or lists of data pointers because the data is identified by PRP1 and PRP2 is invalid. At 214, if PRP2 is invalid, the NVMe command with the modified PRP1 is provided to the SSD for processing.

[0027] If the I / O is greater than 4K, PRP2 is valid, and at 208, it is determined whether PRP2 is a data pointer or a list pointer. For example, if the I / O size is greater than 8K, PRP2 may be a list pointer pointing to a list of data pointers, or if the I / O size is greater than 4K but less than 8K, PRP2 may be a data pointer. If PRP2 is a data pointer, at 216, the address of PRP2 is modified to include an indication associated with the data pointer (such as a data tag), and the VF ID and / or host ID that sent the NVMe command with PRP2. If PRP2 is a list pointer, at 210, it is determined whether there is a memory offset in the host memory where the list starts. If there is no offset, at 212, the address of PRP2 is modified to include an indication associated with the list pointer (such as a PRP list tag) and the VF ID and / or host ID. For example, the higher address bits of the address associated with PRP2 may not be used or reserved, which are replaced by the PRP list tag (when PRP2 is a list pointer) and the host ID. The NVMe switch can store a copy of the replaced bits. Then, at 214, the NVMe command with the modified PRP2 is submitted to the SSD for processing.

[0028] If the offset is not equal to zero, PRP2 may indicate that the list pointer in the host memory may also reference another list pointer. If another list pointer is referenced at 218, processing continues to block 220 to retrieve another list and perform functions similar to 206-218. If no list is referenced at 218, processing continues to blocks 212, 214.

[0029] Determining whether PRP2 is associated with another list pointer may be based on the I / O size and the offset. To illustrate, the list of data pointers may be no larger than 512 data pointers, where each data pointer is 8 bits, such that the list of pointers fits into a 4K buffer associated with PRP2. If each data pointer also points to 4K data, then 2MB of data is associated with PRP2. If the I / O size associated with the command is less than or equal to the sum of 2MB and the 4K associated with PRP1, then the list pointer in PRP2 may not point to another list pointer. If the I / O size is still greater than the non-zero offset, then the list pointer in PRP2 may point to another list pointer.

[0030] The SSD may process an NVMe command received from an NVMe switch. When processing an NVMe command, the SSD may send a PCIe memory request to the NVMe switch together with a PRP entry in the NVMe command. The PCIe memory request may be used to retrieve data associated with a data pointer indicated by a PRP entry, or to retrieve a data pointer associated with a list pointer indicated by a PRP entry. The data pointer or list pointer may point to a memory in a host associated with the NVMe command being processed. In addition, the PRP entry in the PCIe memory request may include a data tag or a PRP list tag and a VF ID and / or a host ID, the data tag or the PRP list tag being used to indicate whether the PRP entry is associated with a data pointer or a list of data pointers, the VF ID and / or the host ID being associated with the NVMe command being processed.

[0031] Figure 3 is an example flow diagram of the functions associated with handling a PCIe memory request from an SSD. The process may be performed by a data processor and may be implemented in hardware, software, firmware, or a combination thereof.

[0032] The PCIe memory request provided by the SSD may have a PRP entry. The PRP entry includes a data tag or a PRP list tag, and a host ID and / or a VF ID that was previously embedded in the PRP entry by the NVMe switch when the NVMe command associated with the PCIe memory request was sent to the SSD. This information can be used to route the PCIe memory request.

[0033] At 302, it is determined whether the PRP entry has a data tag in the address indicating that the PRP entry is associated with a data pointer.

[0034] If the PRP entry has a data tag, then the PRP entry is associated with the data pointer. The host is identified from the host ID or VF ID in the PRP entry, and the bits in the PRP entry that were previously replaced by the NVMe switch are reinserted back into the PRP entry. At 310, a PCIe memory request is routed to the host to perform a PCIe memory request to retrieve the data associated with the data pointer, which is then provided to the SSD in a PCIe memory response received from the host.

[0035] If the PRP entry has a PRP list tag, the PRP entry is associated with a list pointer. The host is identified from the host ID or VF ID in the PRP entry, and the bit in the PRP entry that was previously replaced by the NVMe switch is reinserted back into the PRP entry. At 304, the PCIe memory request is routed to the host so that the host executes the PCIe memory request to retrieve a list of data pointers associated with the list pointer. In some cases, the PRP entry may also include a VF ID, which is then used to further route the PCIe memory request to the VM or virtual host. At 306, the NVMe switch waits for a PCIe memory response to the PCIe memory request, which indicates that a list of pointers is received from the host. Upon receiving this completion, at 308, the pointers in the list of pointers are modified to include data tags, PRP list tags, and / or host IDs in a manner similar to that described in blocks 204, 212, and 216 above to indicate whether each pointer in the pointer points to data or a list of pointers and indicates the host associated with the data pointer or list pointer. In some cases, the PRP entry may include the VFID. The modified list of pointers is then provided to the SSD. When the SSD sends a PCIe memory request to access a pointer, the data tag, PRP list tag, host ID, and / or VF ID are used by the NVMe switch to route the pointer to the host associated with the pointer.

[0036] In some examples, the last entry of the pointer list may be a list pointer that points to a list of another data pointer. The data processor may check every 8 bytes or some other amount of data in the pointer list of the PCI response received from the host. If the data in the list of pointers is a list pointer, the pointer is marked with a PRP list tag. A VF ID and / or host ID is included in the pointer that is associated with the list from which the pointer was retrieved to facilitate the SSD's subsequent retrieval of the data pointed to by the pointer or an additional list of pointers.

[0037] Figure 4An example process for modifying a PRP entry associated with an NVMe command, and retrieving and modifying an associated data pointer and / or list pointer, which is retrieved from the host based on the PRP entry, is illustrated. The PRP entry may be indicated by an NVMe command 402 obtained from the host, in which example, the NVMe command 402 includes a PRP1 data pointer and a PRP2 list pointer. The command processor may modify the PRP1 entry and the PRP2 entry to indicate a tag identifying the host ID / VF ID ("RTNG" or routing) and whether the PRP entry is a data pointer ("DAT") or a list pointer ("PRP"). The modified NVMe command is shown as command 404. The SSD may send a PCIe memory request to the NVMe switch to obtain a list of pointers ("PRP list") from the host. The list of pointers 406 may be obtained by the data processor from the host and modified to indicate whether the pointer is associated with a data pointer ("DAT") or a list pointer ("PRP"). The pointers in the list may also be associated with the host ID and / or VF ID (“RTNG”) from which the list of pointers was obtained. The modified list of pointers is provided as list 408 .

[0038] In some examples, the lists of pointers 406, 408 may include an entry of a list pointer (shown as "Next PRP List Pointer") that points to another list pointer in the host memory. The data processor may detect the list pointer and retrieve a list of pointers associated with the list pointer, which is shown as list of pointers 410. The data processor may modify the pointer to indicate whether the pointer is associated with a data pointer ("DAT") or a list pointer ("PRP"). In addition, the data processor may associate the pointer with a host ID and / or a VF ID ("RTNG") and send the modified list of pointers 412 to the SSD. If the modified list of pointers 412 includes additional list pointers, the data processor may further retrieve the list of pointers associated with the additional list pointers upon request of the SSD in the manner described.

[0039] Example Device

[0040] Figure 5is a simplified block diagram of an NVMe switch 116. The NVMe switch 116 includes a switch processor 502 (which may include multiple processors, multiple cores, multiple nodes, and / or implement multithreading, etc.) that coordinates operations on the NVMe switch. The NVMe switch 116 includes a memory 504. The memory 504 can be a system memory (e.g., one or more of a cache, a random access memory (RAM), a synchronous RAM (SRAM), a dynamic RAM (DRAM), a zero capacitor RAM, a two-transistor RAM, an embedded DRAM (eDRAM), an extended data output RAM (EDO RAM), a double data rate RAM (DDR RAM), an electrically erasable programmable read-only memory (EEPROM), a Nano-RAM (NRAM), a resistive RAM (RRAM), a silicon-oxide-nitride-oxide-silicon memory (SONOS), a parameter random access memory (PRAM), etc.) or any one or more other possible implementations of a non-transitory machine-readable medium / media. In an example, the memory 504 may store a PRP entry 514 having a data tag or a PRP list tag and routing information in the form of a host ID and / or a VF ID associated with an NVMe command received from the host. The memory 504 may also store data associated with a data pointer retrieved from the host and a list of pointers retrieved from the host.

[0041] NVMe switch 116 also includes a bus 506 (e.g., an Advanced Scalable Bus AXI bus). Coupled to bus 506 is an interface 508 that facilitates communication with the non-volatile memory array of the SSD and the host. In this regard, interface 508 may include a PCIe EP cluster and a PCIe RC cluster that provide communication with the host and the SSD, respectively.

[0042] The command processor 516 may be implemented for any of the following, previously described functions: marking a PRP entry associated with an NVMe command using information associated with whether it is a data pointer or a list pointer. The PRP entry may also be modified using the host ID of the physical host that sent the NVMe command and / or the VF ID of the virtual host or VM. The data processor 518 may be implemented for any of the following, previously described functions: routing data pointers and / or list pointers to the host based on the host ID and / or VF ID in the modified PRP entry. These functions may be implemented in hardware and / or software (e.g., computer code, program instructions, program code, computer instructions) stored on a non-transitory machine-readable medium / media. In some cases, the processor 502 and the memory 504 may replace the command processor 510 and the data processor 512 or be in addition to the command processor 510 and the data processor 512 to implement the functions or help implement the functions. In addition, the implementation may include Figure 5 Fewer components or additional components not shown in the figure (e.g., video card, audio card, additional network interface, peripheral devices, etc.). Processor 502 and memory 504 are coupled to bus 506. Although illustrated as coupled to bus 506, memory 504 can be coupled to processor 502.

[0043] Some embodiments have been described in detail above, and various modifications are possible. The disclosed subject matter, including the functional operations described in this specification, can be implemented in electronic circuits, computer hardware, firmware, software, or a combination thereof, such as the structural components disclosed in this specification and their structural equivalents, including programs (such as programs encoded in computer-readable media, which can be storage devices, storage devices, machine-readable storage substrates or other physical, machine-readable media, or a combination of one or more of them) that are potentially operable to cause one or more data processing devices to perform the described operations.

[0044] A program (also referred to as a computer program, software, software application, script or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form including as a stand-alone program or as a module, component, subroutine or other unit suitable for a computing environment. A program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file (e.g., one or more scripts stored in a markup language document) that stores other programs or data, in a single file dedicated to the program in question, or in multiple coordination files (e.g., files storing one or more modules, subroutines or code portions). A program may be deployed to execute on a single computer or on multiple computers located at a station or distributed across multiple stations and interconnected by a communication network.

[0045] Although this specification contains many details, these details should not be interpreted as limitations on the scope of protection that can be claimed, but should be interpreted as descriptions of features specific to specific embodiments. Certain features described in the context of separate embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually in multiple embodiments or in any suitable sub-combination. In addition, although the features may be described above as acting in certain combinations, and even initially claimed as such, one or more features of the claimed combination may be cut out from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variant of the sub-combination.

[0046] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in sequence, or that all of the operations shown be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments.

[0047] Unless specifically stated otherwise, the use of the phrase "at least one of" with the conjunction "and" before a list should not be viewed as an exclusive list and should not be interpreted as a list of categories with one item from each category. A clause reciting "at least one of A, B, and C" may be violated by only one of the listed items, a plurality of the listed items, and one or more of the items in the list and another item not listed.

[0048] Other implementations are within the scope of the following claims.

Claims

1. A method in a storage system, the storage system comprising a switch coupled to two or more hosts and a solid-state drive (SSD), the method comprising: receiving, at the switch, a memory request from the SSD, wherein the memory request includes a physical region page (PRP) entry having: i) an indication that the memory request is associated with a first host among the two or more hosts, and ii) one of: a) a first data tag indicating that the PRP entry is associated with a data pointer, and b) a first PRP list tag indicating that the PRP entry is associated with a list pointer; determining, at the switch, whether the PRP entry includes the first data tag or the first PRP list tag; In response to determining that the PRP entry includes the first data tag, replacing a bit in the PRP entry corresponding to the first data tag with a bit in a PRP entry of a Non-Volatile Memory Express NVMe command corresponding to the memory request from the SSD that was previously replaced by the switch, and routing the memory request to the first host based on the indication that the memory request is associated with the first host; and In response to determining that the PRP entry includes the first PRP list tag, replacing a bit in the PRP entry corresponding to the first PRP list tag with a bit in a PRP entry for an NVMe command corresponding to the memory request from the SSD that was previously replaced by the switch, and The memory request is routed to the first host based on the indication that the memory request is associated with the first host. 2 . The method of claim 1 , wherein receiving the memory request comprises receiving a Peripheral Computer Interface Express (PCIe) read request.

3. The method of claim 1 , wherein the indication that the memory request is associated with the first host is included in upper 2 octets of a Peripheral Computer Interface Express (PCIe) address associated with the PRP entry.

4. The method of any one of claims 1 to 3, wherein the indication that the memory request is associated with the first host comprises a host identifier (ID). 5 . The method of claim 1 , wherein the indication that the memory request is associated with the first host comprises a virtual function (VF) identifier.

6. The method according to any one of claims 1 to 3, further comprising: receiving, at the switch, a list of pointers from the first host in response to routing the memory request to the first host; modifying, at the switch, each pointer in the list of pointers to include: i) an indication that the first host is associated with the list pointer, and ii) one of: a) a second data tag indicating that the pointer points to data, and b) a second PRP list tag indicating that the pointer points to a list pointer; and The list of pointers to the SSD is provided.

7. A non-volatile memory express (NVMe) switch, the NVMe being used in a storage system including two or more hosts and a solid-state drive (SSD), the NVMe switch comprising: Memory; as well as One or more processors configured to: receiving a memory request from the SSD, wherein the memory request includes a physical region page (PRP) entry having: i) an indication that the memory request is associated with a first host among the two or more hosts, and ii) one of: a) a first data tag indicating that the PRP entry is associated with a data pointer, and b) a first PRP list tag indicating that the PRP entry is associated with a list pointer; determining whether the PRP entry includes the first data tag or the first PRP list tag; In response to determining that the PRP entry includes the first data tag, replacing a bit in the PRP entry corresponding to the first data tag with a bit in a PRP entry of a Non-Volatile Memory Express NVMe command corresponding to the memory request from the SSD that was previously replaced by the switch, and routing the memory request to the first host based on the indication that the memory request is associated with the first host; In response to determining that the PRP entry includes the first PRP list tag, replacing a bit in the PRP entry corresponding to the first PRP list tag with a bit in the PRP entry of an NVMe command for the memory request from the SSD that was previously replaced by the switch, and The memory request is routed to the first host based on the indication that the memory request is associated with the first host.

8. The NVMe switch according to claim 7, wherein: The memory request comprises a peripheral computer interface express (PCIe) read request; The NVMe switch includes a PCIe root complex RC cluster coupled to the SSD; as well as The command processor is configured to receive the PCIe read request via the PCIe RC cluster.

9. The NVMe switch according to claim 7, wherein: The NVMe switch includes a PCIe endpoint EP cluster coupled to the two or more hosts; and The command processor is configured to route the memory request to the first host via the PCIe EP cluster.

10. The NVMe switch according to claim 7, wherein: The NVMe switch includes a memory; The one or more processors are configured to execute machine-readable instructions stored in the memory; and The memory stores machine-readable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving the memory request from the SSD, determining whether the PRP entry includes the first data tag or the first PRP list tag, In response to determining that the PRP entry includes the first data tag, replacing a bit in the PRP entry corresponding to the first data tag with a bit in the PRP entry of the NVMe command corresponding to the memory request from the SSD that was previously replaced by the switch, and routing the memory request to the first host based on an indication that the memory request is associated with the first host; wherein the memory further stores machine-readable instructions that, when executed by the one or more processors, cause the one or more processors to, in response to determining that the PRP entry includes the first PRP list tag, replacing a bit in the PRP entry corresponding to the first PRP list tag with a bit in a PRP entry for an NVMe command corresponding to the memory request from the SSD that was previously replaced by the switch, and The memory request is routed to the first host based on the indication that the memory request is associated with the first host.

11. The NVMe switch of any one of claims 7 to 10, wherein the indication that the memory request is associated with the first host is included in the upper 2 octets of a peripheral computer interface express (PCIe) address associated with the PRP entry.

12. The NVMe switch of any one of claims 7 to 10, wherein the indication that the memory request is associated with the first host comprises a host identifier (ID).

13. The NVMe switch of any one of claims 7 to 10, wherein the indication that the memory request is associated with the first host comprises a virtual function (VF) identifier.

14. The NVMe switch according to any one of claims 7 to 10, wherein the one or more processors are further configured to: receiving a list of pointers from the first host in response to routing the memory request to the first host; Modify each pointer in the pointer list to include: i) an indication that the first host is associated with the list pointer, and ii) one of: a) a second data tag indicating that the pointer points to data, and b) a second PRP list tag indicating that the pointer points to a list pointer; as well as Provides a list of pointers to the SSDs.

15. A non-transitory medium storing machine-readable instructions that, when executed by one or more processors, cause the one or more processors to at least perform: receiving a memory request from a solid-state drive (SSD), wherein the memory request includes a physical region page (PRP) entry having: i) an indication that the memory request is associated with a first host among the two or more hosts, and ii) one of: a) a first data tag indicating that the PRP entry is associated with a data pointer and b) a first PRP list tag indicating that the PRP entry is associated with a list pointer; determining whether the PRP entry includes the first data tag or the first PRP list tag; In response to determining that the PRP entry includes the first data tag, replacing a bit in the PRP entry corresponding to the first data tag with a bit in a PRP entry of a Non-Volatile Memory Express NVMe command corresponding to the memory request from the SSD that was previously replaced by the switch, and routing the memory request to the first host based on an indication that the memory request is associated with the first host; and In response to determining that the PRP entry includes the first PRP list tag, replacing a bit in the PRP entry corresponding to the first PRP list tag with a bit in a PRP entry for an NVMe command corresponding to the memory request from the SSD that was previously replaced by the switch, and The memory request is routed to the first host based on an indication that the memory request is associated with the first host.

16. The non-transitory computer readable medium of claim 15, further storing machine readable instructions that, when executed by one or more processors, cause the one or more processors to receive the memory request as a Peripheral Computer Interface Express (PCIe) read request.

17. The non-transitory computer readable medium of claim 15, wherein the indication that the memory request is associated with the first host is included in upper 2 octets of a peripheral computer interface express (PCIe) address associated with the PRP entry.

18. The non-transitory computer readable medium of any one of claims 15 to 17, wherein the indication that the memory request is associated with the first host comprises a host identifier (ID).

19. The non-transitory computer readable medium of any one of claims 15 to 17, wherein the indication that the memory request is associated with the first host comprises a virtual function (VF) identifier.

20. The non-transitory computer readable medium of any one of claims 15 to 17, further storing machine readable instructions that, when executed by one or more processors, cause the one or more processors to: receiving a list of pointers from the first host in response to routing the memory request to the first host; Modify each pointer in the pointer list to include: i) an indication that the first host is associated with the list pointer, and ii) one of: a) a second data tag indicating that the pointer points to data, and b) a second PRP list tag indicating that the pointer points to a list pointer; as well as Provides a list of pointers to the SSDs.

Images