Managed exchange between host and solid-state drive (SSD) based on NVMe protocol

Abstract

Embodiments of the present disclosure relate to managed exchanges between one or more hosts and solid-state drives (SSDs) based on the NVMe protocol for providing host storage device services. A high-speed non-volatile memory (NVMe) switch is located between the host and the storage device. A first storage device access command is received from the host via a high-speed peripheral computer interface (PCIe) to access the storage device. The first storage device access command complies with NVMe, and the storage device includes two or more solid-state drives (SSDs). Based on the first storage device access command, a corresponding second storage device access command is sent to the two or more SSDs. Based on the corresponding second storage device access command, a corresponding completion is received from each of the two or more SSDs. Based on the completion received from each of the two or more SSDs, a completion is sent to the host via a PCIe interface.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to U.S. Provisional Patent Application No. 62 / 716,275, filed on August 8, 2018, entitled “Method and Apparatus for Managed Switching Based on NVMe Protocol,” and U.S. Patent Application No. 16 / 532,081, filed on August 5, 2019, entitled “Managed Switching Between One or More Hosts and Solid State Drives (SSDs) Based on NVMe Protocol for Providing Host Storage Device Services,” which applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to providing storage device services to a host, that is, managing the exchange between one or more hosts and a solid-state device (SSD) based on a high-speed non-volatile memory (NVMe) protocol to provide host storage device services. The host storage device services include virtualization services, performance and fault tolerance services, dual host services and other services. Background Art

[0004] PCIe (Peripheral Component Interconnect Express), officially abbreviated as PCIe or PCI Express, is a high-speed serial computer expansion bus standard. PCIe operates as a motherboard-level interconnect between a host and peripheral components, such as an attached solid-state drive (SSD). The host acts as a server that provides information resources, services, and / or applications to users or other nodes on the network. The host also manages the SSD using a command set defined by non-volatile memory Express (also known as NVMe or NVM Express) for SSD access to obtain storage device services required for operation. The management of the SSD by the host further increases the complexity of the hardware, software, and firmware resident on the host. Summary of the invention

[0005] The present disclosure relates to NVMe switches that are arranged to provide various storage services to a host while reducing the need for the host to directly manage storage in the form of solid-state devices (SSDs). The NVMe switch coordinates the exchange of storage access commands and completions between the host and the SSDs in the storage system to provide these services.

[0006] According to aspects of the described systems and techniques, a method includes: receiving a first storage device access command from a host via a peripheral computer interface express (PCIe) to access a storage device, wherein the first storage device access command complies with a non-volatile memory express (NVMe), and the storage device includes two or more solid-state drives (SSDs); based on the first storage device access command, sending a corresponding second storage device access command to the two or more SSDs; based on the corresponding second storage device access command, receiving a corresponding completion from each of the two or more SSDs; and based on the completion received from each of the two or more SSDs, sending a completion to the host via a PCIe interface.

[0007] According to another aspect of the described systems and techniques, a non-transitory computer-readable medium storing instructions, when the instructions are executed by one or more processors, causes the one or more processors to at least: receive a first storage device access command from a host via a peripheral computer interface express (PCIe) interface to access a storage device, wherein the first storage device access command complies with non-volatile memory express (NVMe), and the storage device includes two or more solid-state drives (SSDs); based on the first storage device access command, send a corresponding second storage device access command to the two or more SSDs; based on the corresponding second storage device access command, receive a corresponding completion from each of the two or more SSDs; and based on the completion received from each of the two or more SSDs, send a completion to the host via the PCIe.

[0008] According to another aspect of the described systems and techniques, a high-speed non-volatile memory express (NVMe) switch includes: a command queue; a completion queue; instructions stored in the NVMe switch, which, when executed by one or more processors of the NVMe switch, cause the NVMe switch to at least: receive a first storage device access command from a host via a high-speed peripheral computer interface (PCIe) interface to access a storage device, wherein the first storage device access command complies with the high-speed non-volatile memory (NVMe), and the storage device includes two or more solid-state drives (SSDs); based on the first storage device access command, send corresponding storage device access commands to the two or more SSDs via the command queue; based on the corresponding second storage device access instruction, receive corresponding completions from each of the two or more SSDs via the completion queue; and based on the completions received from each of the two or more SSDs, send completions to the host via the PCIe interface.

[0009] In this regard, NVMe switches allow reducing the overhead of obtaining various storage device services on the host. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 An example configuration of a storage system is illustrated that allows various storage services to be provided to a host while reducing the need for the host to directly manage storage in the form of a solid state drive (SSD).

[0011] Figure 2 is an example flow chart of the functionality that enables a non-volatile memory express (NVMe) switch to abstract storage devices into VDs.

[0012] Figure 3 is an example flow chart of functionality that enables an NVMe switch to abstract storage devices to provide a certain level of storage device performance and / or storage device fault tolerance to the host.

[0013] Figure 4 is an example flow chart of a function for processing a storage access command when a storage system has two or more hosts.

[0014] Figure 5 is an example flow chart of functions associated with performing NVMe reserved access in an NVMe switch.

[0015] Figure 6 is an example block diagram of an NVMe switch.

[0016] These drawings are only for the purpose of illustrating example embodiments, and it is to be understood that these embodiments are not limited to the arrangements and instrumentalities shown in the drawings. DETAILED DESCRIPTION

[0017] The present disclosure provides examples and details of providing storage device services to the host by managing the exchange between one or more hosts and solid-state devices (SSDs) based on the high-speed non-volatile memory (NVMe) standard. Host storage device services include virtualization services, a certain level of storage device performance and / or storage device fault tolerance, dual host services, and other services. The principles described herein can be applied to control the performance of other types of storage devices, such as hard disk drives (HDDs) or hybrid SSD / HDD drives.

[0018] Figure 1An example configuration of a storage device system 100 is illustrated, which provides various storage device services to the host while reducing the need for the host to directly manage storage devices in the form of solid-state devices (SSDs). The storage device system may include one or more hosts 102, storage devices including one or more solid-state drives (SSDs) 104, and a high-speed non-volatile memory (NVMe) switch 106. One or more hosts 102 can be used as a server that provides information resources, services, and / or applications to users or other nodes on the network. In some examples, the host may include a virtual machine (VM). The VM can be implemented on the host using a high-speed peripheral component interconnect (PCIe) single root input / output virtualization (SR-IOV). SSD 104 is a storage device mechanism based on an integrated circuit (IC) that can permanently store data so that the host 102 can provide the required functions. Compared with a hard disk drive (HDD), SSD 104 can provide higher bandwidth and lower latency. NVMe switch 106 can be placed between host 102 and SSD 104. The NVMe switch 106 processes NVMe commands to control PCIe-based point-to-point switch connections between one or more hosts 102 and SSDs 104 so that each SSD 104 has its own dedicated connection to the one or more hosts 102 .

[0019] The NVMe switch 106 includes a PCIe cluster 110, a management processor 112, a command processor 114, a data processor 116, and a PCIe cluster 108. In some examples, the PCIe cluster 110 may be a PCIe endpoint cluster that facilitates communication between the NVMe switch 106 and two or more hosts 102-104, and the PCIe cluster 108 may be a PCIe root complex cluster that facilitates communication between the NVMe switch 106 and one or more SSDs 104. The command processor 114 processes commands from the host 102. The command may be an NVMe command that obtains data from the SSD 104 and provides it to the host, or stores data from the host in the SSD 104. The command processor 114 may add a tag to the command. The tag may indicate that the command is received from a specific host. Then, the command is sent to the SSD 104. The SSD 104 may send a PCIe memory request, such as a PCIe memory read request, to the NVMe switch based on the command. The PCIe memory request may include a tag that enables the data processor 116 to send the PCIe memory request to the specified host to execute a command associated with the host. The management processor 112 performs management functions for the NVMe switch 106. For example, if the NVMe or PCIe transaction does not fit within what the command processor 114 can execute (e.g., due to an error or the complexity / size of the PCIe memory request), the function may be offloaded to the management processor 112. As another example, the management processor 112 can manage partitions of the SSD 104, reserved access, and namespaces associated with the SSD 104.

[0020] In an example, one or more hosts 102 may each implement a queue pair in accordance with NVMe to facilitate SSD access. The queue pair consists of a submission queue and a completion queue. As shown, one host may have a command queue 118 and a submission queue 120, while another host may have a command queue 122 and a submission queue 124. The host 102 may also have additional queues, such as management queues. The host may generate an NVMe command to obtain data stored in the SSD or store data in the SSD, and the NVMe command is placed in the submission queue. When the command is executed by the SSD, the completion of the execution of the NVMe command may be stored in the completion queue, which is then processed by the command processor 114. Further, the NVMe switch 106 may have a device queue manager 150. The device queue manager 150 provides a queue pair to facilitate the communication of NVMe commands between the NVMe switch 106 and the SSD 104. For each SSD of the SSD 104, the device queue manager 118 has a corresponding submission queue and completion queue. As shown, the first SSD (SSD1) may have a submission queue 126 and a completion queue 128, the second SSD (SSD2) may have a submission queue 130 and a completion queue 132, the third SSD (SSD3) may have a submission queue 134 and a completion queue 136, and the fourth SSD (SSD4) may have a submission queue 138 and a completion queue 140. In this regard, the command processor 114 may terminate NVMe commands associated with access to the SSD 104 from the host 102, and may distribute additional commands to the SSD 104 via queues in the device queue manager 150 to facilitate providing various storage services to the host, such as virtualization services, a certain level of storage performance and / or storage fault tolerance, and access to two or more hosts, as will be further described below. The host 102 does not need to manage the storage system 100 to obtain these services.

[0021] Example virtualized service

[0022] In some examples, an NVMe switch can facilitate virtualization of an SSD as a virtual disk (VD). A VD is a software component that emulates a real physical drive such as an SSD, but looks and behaves like a real physical device. An NVMe switch can also support the creation of a virtual namespace associated with a physical host and a private virtual namespace for a VM.

[0023] Figure 2200 is an example flow chart of functions that enable an NVMe switch to abstract a storage device such as an SSD as a VD. The NVMe switch can provide virtualization services to the host while reducing the need for the host to manage the storage device to obtain these services. The NVMe switch can present the drive to the host. The host can access the device as a physical drive, but the NVMe switch can serve the access as a VD based on two or more SSDs in the SSD by providing aggregation services. The NVMe switch can implement this abstraction in software, hardware, or a combination of hardware and software. In the setup, these functions can be performed by the command processor of the NVMe switch.

[0024] At 202, a first storage device access command, such as an NVMe storage command or an NVMe acquisition command, may be received from a host. The first storage device access command may be to access a drive. The NVMe switch may virtualize multiple SSDs into VDs by terminating the storage device access command from the host and not sending the storage device access command to the SSD.

[0025] Instead, at 204, the NVMe switch may send a corresponding second storage device access command to two or more SSDs associated with the VD. For example, if the VD is associated with four physical SSDs, and the storage device access command from the host needs to access the four physical SSDs, the command processor may send a second storage device access command to each of the four SSDs associated with the VD. The storage device access command is inserted into a submission queue associated with the four SSDs to send the second storage device access command to each of the SSDs.

[0026] At 206, the NVMe switch may receive a respective completion from each of the two or more SSDs, the completion indicating completion of the respective second storage device access command sent to the SSD. For example, the NVMe switch may receive completions from each of the four SSDs. In an example, the completion is inserted into a respective completion queue of the device queue manager associated with the SSD that generated the completion.

[0027] The host can expect to receive a single completion for its storage access command from the storage system. From the host's perspective, access to the VD is the same as access to a single physical drive, even though the VD may be associated with multiple SSDs.

[0028] At 208, based on the completion received from the SSD, the NVMe switch can send a completion to the host via the command processor, in some examples, the completion received from the SSD is a single completion sent to the host. By sending a single completion to indicate completion of the first storage device access command, the drive is virtualized as a single drive to the host that expects to receive a single completion.

[0029] The management processor may also allow capacity to be associated with a VD or the maximum available storage device. The NVMe switch may perform storage device operations requested by the host, but if the storage device operation exceeds the capacity of the VD, the management processor may indicate an error to the host. The management processor may also facilitate the generation of a namespace. Namespace management is the process of defining logical blocks of storage devices assigned to namespaces. A single namespace may typically be associated with a single SSD or VD. The management processor may allow a single namespace to be divided into multiple virtual namespaces associated with a single SSD or VD. For example, the management processor may have an offset table that associates a virtual namespace with different memory offsets in a single namespace. For illustration, namespace 0 may be subdivided into virtual namespaces 1 to 10, each of which is associated with a corresponding memory offset in namespace 0. In this regard, each virtual namespace is associated with a unique plurality of logical blocks in the namespace. The NVMe switch may notify hosts with namespace IDs (NSIDs) of these virtual namespaces, and the host may reserve virtual namespaces using NVMe naming commands. The host may then use the NSID to gain access to the namespace.

[0030] In some examples, the management processor may limit access to the virtual namespace. The virtual namespace may be private or accessible by only one host (if the host includes a VM). The virtual namespace is referred to as a private virtual namespace. Access to the private virtual namespace is limited to one VM because the namespace may be located on an SSD that is also accessed by other VMs or other hosts. Access is limited to prevent a private virtual namespace of a VM from being intentionally or unintentionally written or read by another VM. The management processor may provide a private virtual namespace to the VM via a PCIe virtual function (VF). The PCIe VF may define a logical block associated with a private virtual namespace of the VM, which is then accessed via a storage device access command. Further, in some examples, the quality of service and bandwidth measurement associated with access to the virtual private namespace may be controlled by the NVMe switch through a throttling process because the SSD associated with the virtual private namespace may be accessed by other VMs or hosts. Control of the quality of service and bandwidth measurement for the VM allows ensuring the minimum quality of service and bandwidth for other VMs, and reducing the overhead on the real physical host running the VM.

[0031] Example performance and / or fault tolerance services

[0032] In some examples, the NVMe switch can facilitate controlling storage device performance, such as bandwidth for read / write access to the storage device. The NVMe switch can also facilitate providing fault-tolerant services to the host.

[0033] Figure 3 300 is an example flow chart of functions that enable an NVMe switch to abstract storage to provide a level of storage performance and / or fault-tolerant storage to the host while reducing the need for the host to manage storage devices to obtain these services. Storage performance and / or fault tolerance can take many forms, including SSDs operating as a bunch of flash (JBOF) or a redundant array of independent disks (RAID). The NVMe switch can implement this abstraction in software, hardware, or a combination of hardware and software. In an embodiment, these functions can be performed by a command processor of the NVMe switch.

[0034] At 302, a first storage device access command (such as an NVMe store or get command) is received from a host to access a storage device. The host may expect the storage device to provide a certain performance, such as RAID 0, or a balance of read performance and / or fault tolerance associated with RAID 1 to RAID 10. By terminating the storage device access command from the host and not sending it to the SSD, the NVMe switch can provide these services without the host needing to manage the SSD to achieve read performance and / or fault tolerance.

[0035] Conversely, at 304, the NVMe switch may send a corresponding second storage device access command to two or more SSDs to achieve a certain level of performance and / or fault tolerance for storage device operations associated with the first storage device access command received from the host. In RAID 0 or JBOF, data can be written or read from multiple SSDs in parallel, increasing the bandwidth of storage device operations. The NVMe switch inserts the second storage device access command into the corresponding submission queues of the two or more SSDs to send the second storage device access command to each of the SSDs to read data from the two or more SSDs in parallel and / or write data to the two or more SSDs in parallel. In RAID 1 to RAID 10, data associated with storage device operations can be replicated by mirroring or striping the data on multiple SSDs to improve fault tolerance. In an example, the NVMe switch inserts the second storage device access command into the corresponding submission queues of the two or more SSDs to send the second storage device access command to each of the SSDs to read data from the two or more SSDs in parallel and / or write data to the two or more SSDs associated with mirroring or striping in parallel. When one SSD fails, the second storage device access command can be used to read data by reading data in another drive, and to read or write data in the reconstruction of a new drive when the failed drive is replaced.

[0036] At 306, the NVMe switch may receive a respective completion from each of the two or more SSDs indicating completion of a respective second storage device access command sent to the SSD. For example, the NVMe switch may receive a completion from each of the four SSDs. In the example, the completion is inserted into a respective completion queue in the device queue manager associated with the SSD that generated the completion.

[0037] From the perspective of the host, access to storage devices with expected performance and / or fault tolerance is abstracted to the host. In this regard, the host can expect to receive a single completion for its storage access request from the storage system even if the expected storage performance and / or fault tolerance involves access to multiple SSDs.

[0038] At 308, based on the completion received from the SSD, NVMe may send a completion to the host. The completion sent to the host may be a single completion to indicate the completion of the command sent to the SSD. By sending a single completion to indicate the completion of the first storage device access command, the drive accessed by the host has the expected performance and / or fault tolerance without requiring the host to manage the SSD to obtain these services.

[0039] Example dual host service

[0040] A PCIe connection may include 2, 4, 8, or 16 data transmission lanes. Each transmission lane may include two pairs of lines, one pair for sending and one pair for receiving. In some examples, two or more hosts may be coupled to a PCIe connection and then connected to an NVMe switch. For example, a PCIe connection may be separated into four lanes, and two lanes may be allocated to each host. Further, an SSD coupled to a PCIe switch may have a single port shared between two or more hosts. In this regard, when a storage device system has two or more hosts, an NVMe switch may be responsible for routing data from a single port to an appropriate host to execute a storage device access command. In other examples, two or more hosts may include a VM. An NVMe switch may be responsible for routing data from a single port to a VM to execute a storage device access command.

[0041] Figure 4 400 is an example flowchart of functions for processing storage device access commands when the storage device system has two or more hosts. In some examples, the NVMe switch may also be coupled to a single-port SSD instead of a dual-port SSD. Unlike a dual-port SSD that allows access by two hosts, a single-port SSD may only allow access by one host. This function facilitates associating the storage device access command with the host that sends the storage device access command. The NVMe switch may implement this processing in software, hardware, or a combination of hardware and software. In an example, these functions may be performed by one or more of the data processor and the command processor of the NVMe switch. Further, when the storage device system has two or more hosts, the functions in flowchart 400 may be used to implement virtualization services, control of read / write performance, and fault-tolerant services as described above.

[0042] At 402, a first storage device access command is received from a host. The host may be one of two or more hosts coupled to an NVMe switch, and the NVMe switch receives the first storage device access command. The first storage device access command may be received from a PCIe address associated with the host, wherein each host has its own PCIe address. The two or more hosts may be real physical hosts connected to a PCI port. At 404, based on metadata associated with routing the storage device access command to the NVMe switch, the NVMe switch may identify the host that sent the first storage device access command. The NVMe switch may terminate the first storage device access command from the host. Conversely, in order to execute the first storage device access command, at 406, the NVMe switch may send a second storage device access command with a tag to the SSD, the tag indicating the host that sent the first storage device access command to the NVMe switch. In some examples, the tag may be inserted into the high byte of the NVMe physical region page address. At 408, the NVMe switch may receive a PCIe memory request from the SSD based on a second storage device access command, the second storage device access command having an indication of a host identified in a tag associated with the first storage device access command. For example, the PCIe memory request may be data obtained from a data buffer associated with a read or write command initially received from the host in the host, and the tag may be inserted into the high byte of the NVMe physical region page address in the PCIe memory request. By including the tag, the SSD is able to inform the command processor that the PCIe memory request will be routed to the host identified by the tag. The SSD may be able to process storage device access commands in an order different from the order in which they are sent to the SSD, and the NVMe switch will still be able to identify the host to which the PCIe memory request associated with the storage device access command is sent. At 410, based on the indication of the host, the NVMe switch may route the PCIe memory request to the host. The data processor may perform the routing, and in some examples, the data processor may remove the tag from the PCIe memory request and then route the PCIe memory request to the host.

[0043] In an example, an NVMe switch may support NVMe reservations. A management processor may define a reservation table to coordinate access to a shared namespace by two hosts. The reservation table may indicate whether the host has appropriate permissions to perform a requested operation in the shared namespace. If the host does not have appropriate permissions, the requested operation is denied.

[0044] Figure 5500 is an example flow chart of functions associated with an NVMe switch that performs NVMe reserved access. The NVMe switch can implement this abstraction in software, hardware, or a combination of hardware and software. In an example, these functions can be performed by one or more of a command processor, a management processor, and a data processor of the NVMe switch.

[0045] At 502, the NVMe switch may receive a storage device access command from a host to access a namespace. The host may want to read data from the namespace or write data to the namespace. In some examples, the namespace may be sharable between two or more hosts. Further, each host may have certain access permissions to the shared namespace. For example, one host may be able to read and write to the namespace, while other hosts may only be able to read the data of the namespace. The permissions may be defined by a reservation table, which indicates the permissions associated with each host. The NVMe switch may terminate the storage device access command from the host. At 504, the NVMe switch may determine whether the host has access to the namespace based on the reservation table. Depending on the type of access to the namespace by the host (e.g., write access, read access), and the reservations (e.g., write permissions, read permissions) owned by the host indicated by the reservation table, the management processor may determine whether the storage device access command will be completed or denied. For example, if the host is requesting write access and it has write permission, access is authorized. As another example, if the host is requesting write access and it has read permission, the access is denied. At 506, if the host has access to the namespace, the NVMe switch sends a storage access command to the SSD and sends a completion of the access to the host. In some examples, the completion of the access may be sent to the host when the SSD sends a completion indicating the completion of the storage access command (such as a read or write) to the NVMe switch. At 508, if the host does not have access to the namespace, the NVMe switch sends an error denying access.

[0046] Example Device

[0047] Figure 61 is a simplified example block diagram of an NVMe switch 106 for providing various storage device services to a host while reducing the need for the host to manage solid-state devices (SSDs). The NVMe switch 106 includes a switch processor 602 (which may include multiple processors, multiple cores, multiple nodes, and / or implement multithreading, etc.) that coordinates operations on the NVMe switch 106. The NVMe switch 106 includes a memory 604. The memory 604 can be a system memory (e.g., a cache, a random access memory device (RAM), a static RAM (SRAM), a dynamic RAM (DRAM), a zero-capacitor RAM, a two-transistor RAM, an embedded DRAM (eDRAM), an extended data output RAM (EDO RAM), a double data rate RAM (DDR RAM), an electrically erasable programmable read-only memory device (EEPROM), a nano RAM (NRAM), a resistive RAM (RRAM), a silicon-oxygen-nitrogen-oxygen-silicon memory (SONOS), a parameter random access memory (PRAM), etc.) or any one or more other possible implementations of a non-transitory machine-readable medium / media.

[0048] The NVMe switch 106 also includes a bus 606 (e.g., Peripheral Component Interconnect (PCI), Industry Standard Architecture (ISA), High-Speed ​​PCI, New Bus (NuBus), Advanced Extensible Bus AXI bus, etc.). An interface 608 is coupled to the bus 606, which facilitates communication with the non-volatile memory array of the SSD and the host. In this regard, the interface 608 may include a PCIe EP cluster and a PCIe RC cluster that provide communication to the host and the SSD, respectively.

[0049] While reducing the need for the host to manage the SSD, the management processor 610, data processor 612, and command processor 614 can implement any of the functions previously described, such as VD, RAID, and dual-host functions. These functions can be implemented in hardware and / or software (e.g., computer code, program instructions, program code, computer instructions) stored on non-transitory machine-readable media / media. In some cases, the processor 602 and memory 604 can replace or be in addition to the management processor 610, data processor 612, and command processor 614 to implement or facilitate the implementation of functions. The NVMe switch 106 may further include one or more command and submission queues 616 to facilitate sending NVMe commands to the SSD. Further, the implementation may include fewer or Figure 6 Additional components not shown in the figure (e.g., video card, audio card, additional network interface, peripheral devices, etc.). Processor 602 and memory 604 are coupled to bus 606. Although shown as being coupled to bus 606, memory 604 can be coupled to processor 602.

[0050] A small number of implementations have been described in detail above, and various modifications are possible. The disclosed subject matter, including the functional operations described in this specification, can be implemented in electronic circuitry, computer hardware, firmware, software, or a combination thereof, such as the structural components disclosed in this specification and their structural equivalents: including a program potentially operable to cause one or more data processing devices (such as a processor) to perform the described operations, such as a program encoded in a non-transitory computer-readable medium, which can be a memory device, a storage device device, a machine-readable storage device substrate, or other physical machine-readable medium, or a combination of one or more of them.

[0051] A program (also referred to as a computer program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it may be deployed in any form, including as a stand-alone program or as a stand-alone module, component, subroutine, or other unit suitable for use in a computing environment. A program may be stored as part of a file with other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A program may be deployed for execution on one computer or on multiple computers, located at one site or distributed across multiple sites and interconnected by a communications network.

[0052] Although this specification includes a number of details, these should not be interpreted as limitations on the scope of the requirements, but as an illustration of features that may be specific to a particular implementation. Certain features described in this specification may also be combined in a single implementation in the context of a separate implementation. On the contrary, the various features described in the context of a single implementation may also be implemented separately in multiple implementations or in any appropriate sub-combination. In addition, although features may act in certain combinations as described above, and even in some cases, one or more features in the combination required may be removed from the combination as originally required, and the combination required may point to a sub-combination or a deformation of a sub-combination.

[0053] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above implementations should not be understood as requiring such separation in all implementations.

[0054] The use of the phrase "at least one of" in conjunction with "and" preceding a list should not serve as an exclusive list and should not be interpreted as a category list using one item from each category unless specifically noted. Statements referencing "at least one of A, B, and C" can be infringed by only one of the listed items, a plurality of the listed items, and one or more of the items in the list and additional items not listed. Other implementations are within the scope of the following claims.

Claims

1. A method implemented by a switch configured to communicate with a plurality of hosts and two or more solid-state drives (SSDs), the method comprising: Receiving a first storage device access command from a first host via a high-speed peripheral computer interface (PCIe) interface of the switch to access the two or more SSDs, wherein the first storage device access command complies with a high-speed non-volatile memory (NVMe) protocol; Based on the first storage device access command, sending a corresponding second storage device access command to the two or more SSDs; receiving a plurality of first completions from the two or more SSDs based on the corresponding second storage device access commands; as well as sending a single second completion to the first host via the PCIe interface based on the plurality of first completions received from the two or more SSDs; receiving, at the switch, a third storage device access command from one of the two or more SSDs, the third storage device access command including an indication that the third storage device access command corresponds to the first host; determining, by the switch, that the third storage device access command corresponds to the first host using the indication included in the third storage device access command; removing, by the switch, the indication from the third storage device access command; as well as After removing the indication from the third storage device access command, the switch sends the third storage device access command to the first host.

2. The method of claim 1 , wherein sending the corresponding second storage device access commands to the two or more SSDs comprises: The corresponding second storage device access commands are sent to write the same data to the two or more SSDs in parallel.

3. The method according to claim 1, further comprising: determining that the plurality of first completions from the two or more SSDs indicate that the corresponding second storage device access commands are completed by the two or more SSDs; wherein in response to determining that the plurality of first completions from two or more SSDs indicate that the corresponding second storage device access command is completed by two or more SSDs, sending the single second completion to the host via the PCIe interface.

4. The method according to claim 1, further comprising: determining, at the switch, that the first storage device access command is received from the first host; as well as inserting, by the switch, into each of the corresponding second storage device access commands, an indication that the second storage device access command corresponds to the first host; After the indication is inserted into the corresponding second storage device access command, the corresponding second storage device access command is sent to the two or more SSDs.

5. The method of claim 1 , wherein sending the corresponding second storage device access commands to the two or more SSDs comprises: The respective second storage device access commands are sent to the two or more SSDs via respective submission queues associated with the two or more SSDs, respectively.

6. The method of claim 1 , wherein receiving the respective plurality of first completions from the two or more SSDs comprises: The plurality of first completions are received from the two or more SSDs via respective completion queues associated with the two or more SSDs.

7. The method according to claim 1, wherein: The first storage device access command corresponds to a namespace; In response to determining that the first host has access to the namespace, sending a plurality of the corresponding second storage device access commands to the two or more SSDs; and The method includes denying the first storage access command in response to determining that the first host does not have access to the namespace.

8. A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a switch configured to communicate with a plurality of hosts and two or more solid-state drives (SSDs), cause the one or more processors to at least: controlling the switch to receive a first storage device access command from a first host among the plurality of hosts to access the two or more SSDs, wherein the first storage device access command complies with a high-speed non-volatile memory (NVMe) protocol, and wherein the first storage device access command is received via a high-speed peripheral computer interface (PCIe) of the switch; Controlling the switch to send a corresponding second storage device access command to the two or more SSDs based on the first storage device access command; controlling the switch to receive a corresponding plurality of first completions from the two or more SSDs based on the corresponding second storage device access commands; controlling the switch to send a single second completion to the first host via the PCIe based on the plurality of first completions received from the two or more SSDs; controlling the switch to receive a third storage device access command from one of the two or more SSDs, the third storage device access command including an indication that the third storage device access command corresponds to the first host; using the indication included in the third storage device access command, determining that the third storage device access command corresponds to the first host; removing the indication from the third storage device access command; and After removing the indication from the third storage device access command, controlling the switch to send the third storage device access command to the first host.

9. The non-transitory computer-readable medium of claim 8, wherein the instructions to send the corresponding second storage device access commands to the two or more SSDs comprise: Instructions to send the corresponding second storage device access commands to write the same data to the two or more SSDs in parallel.

10. The non-transitory computer readable medium of claim 8, further comprising: determining the plurality of first completions from the two or more SSDs indicating that a plurality of the corresponding second storage device access commands were completed by the two or more SSDs; Wherein controlling the switch to send the instruction for the single second completion includes: controlling the switch to send the instruction for the single second completion in response to determining that multiple first completion indications from the two or more SSDs corresponding to the second storage device access commands are completed by the two or more SSDs.

11. The non-transitory computer readable medium of claim 8, further storing instructions that, when executed by one or more processors of the switch, cause the one or more processors to: determining that the first storage device access command is received from the first host; and An indication that the second storage device access command corresponds to the first host is inserted into each of the corresponding second storage device access commands.

12. The non-transitory computer-readable medium of claim 8, wherein the instructions for controlling the switch to send the corresponding second storage device access commands to the two or more SSDs comprise: Instructions for controlling the switch to send the corresponding second storage device access commands to the two or more SSDs via corresponding submission queues associated with the two or more SSDs, respectively.

13. The non-transitory computer readable medium of claim 9, wherein the instructions to control the switch to receive the plurality of first completions from the two or more SSDs comprise: The first plurality of completed instructions are received from the two or more SSDs via respective completion queues associated with the two or more SSDs.

14. The non-transitory computer-readable medium according to claim 8, wherein the first host comprises a virtual machine VM, the non-transitory computer-readable medium further comprising: An instruction to control the switch to provide a virtual private namespace to the VM.

15. A high-speed non-volatile memory (NVMe) switch for communicating with a plurality of hosts and two or more solid-state drives (SSDs), the plurality of hosts comprising a first host and a second host, the NVMe switch comprising: Multiple submission queues; Multiple completion queues; An electronic circuit configured to control the NVMe switch to at least perform the following operations; Receiving a first storage device access command from the first host via a high-speed peripheral computer interface PCIe interface to access the two or more SSDs, wherein the first storage device access command complies with a high-speed non-volatile memory NVMe protocol; Based on the first storage device access command, sending a corresponding second storage device access command to the two or more SSDs via the plurality of submission queues; receiving a plurality of first completions from the two or more SSDs via respective completion queues based on the respective second storage device access instructions; as well as sending a single second completion to the first host via the PCIe interface based on the plurality of first completions received from the two or more SSDs; receiving a third storage device access command from one of the two or more SSDs, the third storage device access command including an indication that the third storage device access command corresponds to the first host; using the indication included in the third storage device access command, determining that the third storage device access command corresponds to the first host; removing the indication from the third storage device access command; and After removing the indication from the third storage device access command, the third storage device access command is sent to the first host.

16. The NVMe switch of claim 15, wherein the storage device access command is a storage request or a get request.

17. The NVMe switch of claim 15, wherein corresponding submission queues are respectively associated with the two or more SSDs.

18. The NVMe switch of claim 15, wherein corresponding completion queues are respectively associated with the two or more SSDs.

Images