Techniques for initializing a remotely accessible storage device

Abstract

Device (500) comprising: Circuits (520) for a network input / output, I / O, device (1200); a parameter module (522-1) for execution by the circuitry (520) to receive one or more parameters (505) to enable the network I / O device (1200) to connect to a remote server via a network communication link; a connection module (522-2) for execution by the circuitry (520) to connect to the remote server using the one or more parameters (505); a path module (522-3) for execution by the circuitry (520) to establish a control path to a non-volatile memory express, NVMe, controller managed at the remote server using a remote direct memory access, RDMA, protocol; and a receive module (522-6) for execution by the circuitry (520) to receive an RDMA service tag, STag, (515) from the NVMe controller and to access an operating system, OS, kernel and one or more device drivers.

Description

TECHNICAL FIELD

[0001] Examples described here generally refer to the initialization of a device (e.g., “booting”) from a remote storage device. BACKGROUND

[0002] The ability to initialize or boot a computing device from a network can be used in some computing resource deployments. In some examples, booting from a network can enable client devices to be deployed and operated without a hard disk or solid-state drive. The absence of a hard disk or solid-state drive can enable reduced costs, easier maintenance management, and greater consolidation of storage resources. Furthermore, it can be easier to securely lock down a networked client or host device without an associated storage device.

[0003] According to some examples, client or host devices can boot remotely by loading their operating system (OS) image via a connection to a storage server over a network. Remote booting over the network can enable flexible deployment. Flexibility can be provided in part by the storage server centrally controlling the OS and its configuration, which can be loaded on a given remotely booted client device.

[0004] US 8966 172 B2 describes a system, method, and / or apparatus for processor-independent data storage in a PCIE-based shared storage environment. In one aspect, a method comprises processing a memory-based request received at an adapter circuit of a controller connected to a disk array to direct the memory-based request to at least one processor of the disk array and a plurality of storage devices of the disk array.The method also includes forwarding the data request in the other format compatible with the storage device via an interface circuit of the control device directly to at least one storage device of the plurality of storage devices of the disk array connected to the control device, which is independent of a processor of the disk array, to store data associated with the data request based on a mapping table.

[0005] US 2005 / 0071623 A1 describes a system and method for quickly and efficiently transferring data over a network to a processing system during a pre-boot runtime of the processing system. During a pre-boot runtime of the processing system, a plurality of data packets are received over a network. Each of the plurality of data packets contains one of a corresponding plurality of data segments. The plurality of data packets are parsed using a network protocol stack to extract the plurality of data segments during the pre-boot runtime. A portion of the network protocol stack is executed in a hardware unit of the processing system. The majority of the data segments are transferred to the system memory of the processing system during the pre-boot runtime.

[0006] US 2008 / 0 313 364 A1 discloses an apparatus, system, and method for sharing a device among multiple hosts. The apparatus, system, and method include an RDMA setup module and an RDMA execution module. The RDMA setup module prepares a solid-state storage controller for an RDMA operation to transfer data of a file or object between the solid-state storage controller and a requesting device in response to a storage request. The storage request may be substantially free of data, and the solid-state storage controller may control a solid-state storage device via a storage input / output ("I / O") bus. The solid-state storage controller controls the storage of data in the solid-state storage device, and the requesting device is connected to the solid-state storage controller via a computer network.The RDMA execution module performs the RDMA operation to transfer the data between the requesting device and the solid-state storage controller.

[0007] US 8,700,724 B2 describes systems and methods that enable one-time remote direct memory access (RDMA). In one embodiment, a system that transmits data over an RDMA network may include, for example, a host. The host may include, for example, a driver and a network interface card (NIC), with the driver coupled to the NIC. The driver and the NIC may perform a one-time initiation process and / or a one-time completion process of an RDMA operation. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 illustrates a first exemplary system. Fig. Figure 2 illustrates an example completion queue element. Fig. Figure 3 illustrates a second exemplary system. Fig. Figure 4 illustrates a remote boot scheme. Fig. 5 illustrates an exemplary block diagram for a first device. Fig. 6 illustrates an example of a first logic flow. Fig. 7 illustrates an example of a first storage medium. Fig. Figure 8 illustrates an example of a first network input / output (I / O) device. Fig. 9 illustrates an exemplary block diagram for a second device. Fig. 10 illustrates an example of a first logic flow. Fig. 11 illustrates an example of a second storage medium. Fig. Figure 12 illustrates an example of a second network I / O device. SUMMARY OF THE INVENTION

[0008] The object underlying the invention is to provide a device and a method that enable client or host devices to be booted remotely over a network, thus enabling flexible use. This object is achieved by the subject matter of the independent claims. Further advantageous embodiments are specified in the subclaims. DETAILED DESCRIPTION

[0009] As described in the present disclosure, remote booting may enable flexible deployment because the OS on a given client device can be controlled by the storage server, through which the given client device can be remotely booted. Recently, storage servers have begun to include both network input / output (I / O) devices and storage controllers that have enhanced capabilities and attempt to minimize the involvement of the operating system and host processor. For example, hardware elements such as command entry and command execution queues may be used by a storage server's network input / output (I / O) device and storage controller to enable a client or host device to remotely access memory through a method known as remote direct memory access (RDMA).

[0010] Storage controllers managed by these remotely accessible storage servers are also designed to operate in accordance with relatively new interconnect communication protocols that work well with RDMA. Furthermore, these storage controllers can control access to hard disk drives (HDDs) or solid-state drives (SSDs). The SSDs can include, but are not limited to, various types of non-volatile memory, such as 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon (SONOS) memory, polymer memory, nanowire, ferroelectric transistor random access memory (FeTRAM or FeRAM), nanowire, or electrically erasable programmable read-only memory (EEPROM).In some examples, accessing HDDs or SSDs may involve using interconnect communication protocols described in industry standards or specifications (including successors or variants) such as the Peripheral Component Interconnect (PCI) Express Base, Revision 3.0, published in November 2010 (“PCI Express” or “PCIe”) and / or using types of controller interfaces for PCIe-based SSDs such as the Non-Volatile Memory Express (NVMe) specification, Revision 1.1, published in October 2012.

[0011] Storage controllers that operate in accordance with the NVMe specification ("NVMe controllers") can minimize operating system and host processor involvement when allowing a remote client or host device to access storage devices such as SSDs. These types of remotely accessible NVMe-controlled storage devices, when coupled with a high-speed network connection (e.g., 10 gigabits per second (Gbps) or higher), can enable a relatively rapid boot time compared to traditional types of on-premises rotating storage media. However, storage devices such as SSDs configured to work with NVMe controllers are often expensive, and there is a strong incentive to consolidate client usage and optimize the hardware provisioning of these types of storage devices. Due to these and other challenges, the examples described here are necessary.

[0012] In some examples, techniques may be implemented to initialize or boot a client or host device from a remotely accessible storage device. For these examples, circuitry for an NW I / O device coupled to a host device may be capable of executing various modules to facilitate remote booting of the host device. The various modules may include a parameter module to receive one or more parameters to enable the NW I / O device to connect to a remote server (e.g., a storage server) via a network communication link. The various modules may also include a connection module to connect to the remote server using the one or more parameters, and a path module to establish a control path to an NVMe controller managed at the remote server using an RDMA protocol.

[0013] According to some examples, the various modules for execution by the circuitry for the NW I / O device may also include a receive module to receive an RDMA Service Tag (STag) from the NVMe controller, representing an allocated portion of a storage device controlled by the NVMe controller that is accessible using the RDMA STag. In some examples, the allocated portion may store an OS kernel and one or more device drivers. The various modules may also include a store module to store the RDMA STag in non-volatile memory at the NW I / O device that is accessible to a system basic input / output system (BIOS) to enable the system BIOS to use the RDMA STag to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the host device.

[0014] Fig. 1 illustrates a first exemplary system. As shown in Fig. 1, the first example system includes a system 100 having a client 10 communicatively coupled to the server 20 via the network 50. According to some examples, the terms "host computer," "host device," "host," "client device," "client," and "network node" may be used interchangeably and may mean, for example, without limitation, one or more end stations, mobile internet devices, smartphones, media devices, I / O devices, tablet computers, appliances, intermediaries, network interfaces, clients, and / or portions thereof. Furthermore, in some examples, the terms "server," "storage server," or "remote server" may be used interchangeably and may mean, for example, without limitation, a server that can be accessed remotely (e.g., via a network connection) by a "host computer," a "host device," "host," "client device," "client," "network node," and a "node."Although client 10, server 20, and network 50 are referred to in the singular, it should be understood that each such corresponding component may include a plurality of such corresponding components without departing from these examples.

[0015] According to some examples, a "network" may be or include any mechanism, device, modality, and / or portions thereof that allows, facilitates, and / or enables two or more entities to be communicatively coupled to one another. In some examples, a first entity may also be "communicatively coupled" to a second entity if the first entity is capable of sending and / or receiving one or more commands and / or data to the second entity. Furthermore, data and information may be used interchangeably and may be or include one or more commands (e.g., one or more program instructions), and / or one or more such instructions may be or include data and / or information. For these examples, a "command" may also include data and / or one or more commands.

[0016] Client 10 may include RDMA-capable network interface controllers (RNICs), referred to herein as NW I / O devices 106, and / or one or more (and in the example shown in Fig. 1 a plurality of) buffers 13.

[0017] As shown in Fig. 1, server 20 may include one or more integrated circuit (IC) chips 180, memory (MEM) 21, and / or storage 150. One or more chips 180 may include the circuitry 118, which may include an NW I / O device 108, a manageability module 109, or an NVMe controller 112. Although not shown in Fig. 1, in some examples, the NW I / O device 108 and / or the NVMe controller 112 may be separately attachable devices that couple to the server 20 and include circuitry as described in more detail below.

[0018] Also as shown in Fig. 1, the one or more chips 180 may be integrated within one or more multi-core host processors (HP) and / or central processing units (CPU) 12. Although not shown in the figures, server 20 may also include one or more chipsets or devices (to include, among other things, memory or input / output controller circuitry). NW I / O device 108, NVMe controller 112, and / or HP / CPU 12 may be capable of communicating with each other. Additionally, NW I / O device 108, NVMe controller 112, manageability module 109, and / or HP / CPU 12 may be capable of accessing and / or communicating with one or more other components of server 20 (such as memory 21 and / or storage 150) via one or more such chipsets. In some examples, client 10 and / or NW I / O device 106 may be located remotely (e.g., geographically remotely) from server 20 and / or NW I / O device 108.In other examples, client 10 and / or NW I / O device 106 may be located somewhat closer to server 20 and / or NW I / O device 108, with both being within a server rack or even on the same chip in a microserver configuration.

[0019] According to some examples, "circuits" may include, for example, individual (or in any combination) analog circuits, digital circuits, wired circuits, programmable circuits, coprocessor circuits, state machine circuits, and / or memory, which may include program instructions that can be executed by programmable circuits. Furthermore, in some examples, a processor, HP, CPU, processor core (PC), core, and controller may each include corresponding circuits capable of executing one or more arithmetic and / or logical operations and / or one or more instructions. An integrated circuit chip may include one or more microelectronic devices, substrates, and / or dies. Although not shown in Fig. 1, server 20 may include a graphical user interface system, including, for example, a corresponding keyboard, pointing device, and display device that allows a human user to enter commands into server 20 and / or system 100 and monitor the operation thereof. Furthermore, memory may include one or more of the following types of memory: semiconductor firmware memory, programmable memory, non-volatile memory, read-only memory, electrically programmable memory, random access memory, flash memory, disk memory, optical disk memory, and / or other or later developed computer-readable and / or writable memory.

[0020] In some examples, storage 150 may include mass storage 156. For these examples, storage 150 may include one or more devices in which data may be stored and / or retrieved. Also, for these examples, mass storage may include memory capable of non-volatile data storage. For example, mass storage 156 may include, without limitation, one or more non-volatile electromechanical, magnetic, optical, and / or solid-state storage devices. These devices may include hard disk drives (HDDs) or solid-state drives (SSDs).The SSDs can include non-volatile types of memory such as 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon (SONOS) memory, polymer memory, nanowire, ferroelectric transistor random access memory (FeTRAM or FeRAM), nanowire, or electrically erasable programmable read-only memory (EEPROM).

[0021] According to some examples, manageability module 109, NVMe controller 112, memory 150, or storage 156 may be capable of operating in accordance with the PCIe specification and / or the NVMe specification.

[0022] One or more machine-readable program instructions may be stored in memory 21. During operation of server 20, these machine-readable instructions may be accessed and executed by one or more host processors 12, NW I / O device 108, and / or NVMe controller 112. When executed by one or more HPs 12, these one or more machine-readable instructions may result in one or more operating system environments (OSE) 32 being executed by one or more HPs 12 and becoming memory-resident in memory 21.When these machine-readable instructions are executed by the NW I / O device 108 and / or the NVMe controller 112, these one or more instructions may also result in one or more command interfaces 110 of the NVMe controller 112, one or more doorbells 192, one or more pointers 202, one or more agents 194, one or more completion queues 124, and / or one or more submission queues 126 being established and / or executed by the NW I / O device 108 and / or NVMe controller 112, and / or they become memory resident in the memory 21.

[0023] According to some examples, one or more OSEs 32 may include one or more operating systems (OSs) 31 and / or one or more NW I / O devices and / or NVMe controller drivers 33. These one or more drivers 33 may be different from one or more OSs 31. Alternatively or additionally, without departing from these examples, one or more corresponding portions of one or more OSs 32 and / or drivers 33 may not be different from one another and / or may be included within one another. Likewise, without departing from these examples, circuitry 118, NW I / O device 108, manageability module 109, and / or NVMe controller 112 may be different from one another or, alternatively, may be included within the one or more unillustrated chipsets and / or within HP 12.Also, without departing from these examples, one or more portions of memory 21 may be included in or managed by NW I / O device 108, manageability module 109, NVMe controller 112, circuitry 118, HP 12, and / or IC 180.

[0024] In some examples, a portion or subset of an entity may include all or less than all of the entity. Furthermore, for these examples, a method, daemon, program, driver, operating system, application, kernel, and / or virtual machine monitor may each (1) include and / or (2) result in and / or from the execution of one or more operations and / or program instructions.

[0025] According to some examples, a command interface may at least partially facilitate, permit, and / or implement the exchange, transmission, and / or reception of data and / or one or more commands. For these examples, a queue, a buffer, and / or a doorbell may be one or more locations (e.g., specified and / or indicated by one or more addresses) in memory in which data and / or one or more commands may be at least temporarily stored. Furthermore, a queue element may include data and / or one or more commands to be stored and / or stored in one or more queues, such as one or more descriptors and / or one or more commands. Additionally, for these examples, a pointer may indicate, address, and / or specify one or more locations and / or one or more elements in memory.

[0026] In some examples, NW I / O device 106 and NW I / O device 108 may exchange data and / or commands over network 50 according to one or more protocols that may conform to and / or be compatible with an RDMA protocol, such as Internet Wide Area RDMA Protocol (iWARP), Infiniband (IB) protocol, Ethernet protocol, TCP / IP protocol, and / or RDMA over Converged Ethernet (RoCE) protocol. For example, the iWARP protocol may conform to and / or be compatible with Recio et al., "An RDMA Protocol Specification," Internet Draft Specification, Internet Engineering Task Force (IETF), Oct. 21, 2002. For example, the Ethernet protocol may also conform to and / or be compatible with the Institute of Electrical and Electronics Engineers, Inc. (IEEE) Std. 802.3-2008, December 26, 2008.Additionally, for example, the TCP / IP protocol may conform to and / or be compatible with the protocols described in Internet Engineering Task Force (IETF) RFCs (RFC) 791 and 793, published in September 1981. Furthermore, the IB protocol may conform to and / or be compatible with the Infiniband™ Architecture Specification, Vol. 2, Rev. 1.3, published in November 2012. Additionally, for example, the RoCE protocol may conform to and / or be compatible with the Infiniband Architecture Specification Supplement, Vol. 1, Rev. 1.2.1, Appendix A16: "RDMA over Converged Ethernet (RoCE)," published in April 2010. Many different, additional, and / or other protocols may be used for this data and / or command exchange without deviating from these examples (e.g., earlier and / or later developed versions of said related and / or other protocols).

[0027] According to some examples, circuitry 118 may permit and / or facilitate access by NW I / O device 106 via NW I / O device 108 from one or more command interfaces 110. For example, circuitry 118 may permit and / or facilitate NW I / O device 106 accessing one or more command interfaces 110 in a manner independent of OSE 32 in server 20. This accessing may include, for example, writing at least one queue element (e.g., one or more queue elements (QE) 116) to one or more submission queues (SUB Q) 114 in one or more command interfaces (CMD I / F) 110. This may cause the NW I / O device 108 to perform one or more operations involving memory 150 and / or storage 156 connected to the NVMe controller 112 to forward commands to the NVMe controller 112.The NVMe controller 112 may perform these one or more operations in response to the one or more queue items 116 (e.g., after and in response to the one or more queue items 116 being written to one or more submission queues 114 and then forwarded by the NVMe I / O device 108). These one or more operations involving storage 150 and / or storage 156 may include one or more write operations and / or one or more read operations involving storage 150 and / or storage 156. For these examples, client 10 may therefore be able to access storage 150 and / or storage 156 via the one or more read operations and / or one or more write operations performed by the NVMe controller 112.

[0028] For example, during operation of system 100, client 10 and / or NW I / O device 106 may authenticate client 10 and / or NW I / O device 106 with server 20 and / or logic and / or properties at NW I / O device 108. This may result in client 10 and / or NW I / O device 106 being granted permission to access devices managed at or controlled by elements of server 20 (e.g., via NW I / O device 108). At the same time, before or after this, NW I / O device 108, NVMe controller 112, one or more agents 194, and / or OSE 32 in memory 21 may generate, establish, and / or manage one or more interfaces 110 and / or one or more indicators 181. The one or more interfaces 110 and / or indicators 181 may indicate where in the memory 21 (e.g., one or more locations) one or more interfaces 110 and / or the components thereof are located.For example, one or more indicators 181 may indicate one or more locations in memory 21 where one or more submission queues 114, one or more completion queues 120, one or more doorbells 170, and / or one or more buffers 130A...130N are located. The network I / O device 108 may provide one or more indicators 181 to the network I / O device 106 via the network 50. Thereafter, the network I / O device 106 may use one or more of the one or more indicators 181 to access one or more command interfaces 110 and / or one or more components of the one or more command interfaces 110. One or more indicators 181 may include one or more handles (e.g., assigned to transaction contexts) for one or more zones in memory 21, such as, in this example, one or more STags that may correspond to and / or be compatible with an RDMA protocol (e.g., iWARP, IB, RoCE).In some examples, the one or more zones in memory 21 may be included in one or more buffers that are managed to facilitate remote access to memory 150 or storage 156 by client 10.

[0029] After receiving one or more indicators 181, client 10 and / or NW I / O device 106 may issue one or more commands 105 to server 20 via network 50 and NW I / O device 108 to NVMe controller 112 in a manner that bypasses and / or is independent of OSE 32's involvement. The one or more commands 105 may instruct NVMe controller 112 to perform one or more operations involving memory 150 and / or storage 156.

[0030] According to some examples, one or more commands 105 may conform to and / or be compatible with an RDMA protocol (e.g., iWARP, IB, RoCE). One or more commands 105 may include and / or specify one or more queue elements 116 that may embody and / or indicate the one or more commanded operations involving memory 150 and / or storage 156. Although not shown in Fig. 1, one or more instructions 105 may include, specify, and / or indicate one or more of the indicators 181 that may indicate one or more locations in one or more submission queues 114 as one or more intended destinations of one or more queue elements 116.

[0031] In some examples, one or more queue elements 116 may result in NW I / O device 108 forwarding a command such that NVMe controller 112 performs or executes one or more write operations involving memory 150 and / or storage 156. Therefore, one or more commands 105 may also include and / or specify that data 199 be written as a result of NW I / O device 108 forwarding one or more queue elements 116 to NVMe controller 112, memory 150, and / or storage 156. One or more commands 105 may include, specify, and / or indicate one or more of the indicators 181, which may indicate one or more locations of one or more buffers (e.g., buffer(s) 13) to which the data 199 is to be written (at least temporarily) to a client 10.

[0032] In some examples, in response to receiving one or more commands 105, the NW I / O device 108 may directly write (e.g., according to RDMA protocol (e.g., iWARP, IB, RoCE) and / or in a manner that bypasses and / or is independent of OSE 32) to one or more submission queues 114 and one or more buffers 130A in the manner instructed by one or more commands 105, one or more queue elements 116, and the data 199. Therefore, by issuing one or more commands 105 to the NW I / O device 108, the NW I / O device 106 may actually write one or more queue elements 116 and the data 199 to one or more submission queues 114 and one or more buffers 130A, respectively.

[0033] One or more commands 105 may also include and / or specify one or more values ​​201 and one or more of the indicators 181 that may indicate one or more locations of one or more doorbells 170 to which one or more values ​​201 may be written. In response to these one or more values ​​201 and these one or more of the indicators 181 in one or more commands 105, NW I / O device 108 may write one or more values ​​201 to the doorbell 170 directly (e.g., according to RDMA protocol (e.g., iWARP, IB, RoCE) and / or in a manner that bypasses and / or is independent of the OSE 32) in the manner instructed by one or more commands 105. Writing one or more values ​​201 to the doorbell 170 may cause the doorbell 170 to ring. Therefore, by issuing one or more

[0034] Commands 105 to the NW-I / O device 108 cause the NW-I / O device 106 to actually ring the doorbell 170.

[0035] According to some examples, ringing a doorbell associated with an entity may include and / or involve writing one or more values ​​to one or more memory locations (e.g., associated with, comprising, and / or embodying the doorbell), which may result in the entity performing and / or triggering one or more operations and / or actions. In some examples, doorbells 170 and / or 192 may appear to CPU 12 and / or server 20 as one or more corresponding memory locations (not shown) in corresponding memory (not shown) in NVMe controller 112 and / or NW I / O device 108.

[0036] In some examples, in response to the ringing of doorbell 170, NVMe controller 112 may return to a fully operational state (e.g., if NVMe controller 112 previously entered a reduced power state relative to that fully operational state) and read one or more queue items 116 written to one or more submission queues 114. NVMe controller 112 may then execute the one or more commands specified and / or embodied by one or more queue items 116. This may result in NVMe controller 112 performing the one or more operations (e.g., one or more of the writes to memory 150 and / or storage 156 from the data 199 stored in one or more buffers 130A) involving memory 150 and / or storage 156.

[0037] According to some examples, after completing one or more of these operations involving memory 150 and / or storage 156, the NVMe controller 112 may write one or more completion queue elements (CQE) 129 to one or more completion queues 124. After completing one or more of these operations involving memory 150 and / or storage 156, the NVMe controller 112 or the manageability module 109 may also write one or more values ​​to one or more doorbells 192 connected to the NW I / O device 108. This may ring one or more doorbells 192. In response to the ringing of one or more doorbells 192, the NW I / O device 108 (e.g.,via one or more RDMA write operations) write one or more completion queue elements 190 to one or more completion queues 120 and then forward the one or more completion queue elements 190 to one or more buffers 13 in client 10 (e.g., via one or more responses 197).

[0038] In some examples, after one or more (e.g., multiple) such write and / or read operations involving memory 150 and / or mass storage 156 have been performed, one or more agents 194 may perform certain management functions. For example, one or more agents 194 may specify one or more submission queue entries / items 196A...196N in one or more submission queues 126 associated with the NW I / O device 108 and / or one or more submission queue entries / items QE A...QE N in the table 250 (see Fig. 2). As described further below, these elements 196A...196N and / or QE A...QE N, when executed by NW I / O device 108, may permit and / or facilitate the copying or forwarding of one or more other queue entries (e.g., one or more NVMe controller 112 completion entries 129) to client 10 and / or NW I / O device 106 and / or the data read from NVMe controller 112.

[0039] According to some examples, these management functions may also include updating (e.g., advancing accordingly) by one or more agents 194 of one or more pointers (e.g., the ring pointers PNTR 202) associated with one or more queue pairs (e.g., submit / complete queue pair 114, 120 and / or submit / complete queue pair 126, 124) associated with NW I / O controller 108 and NVMe controller 112. This may allow new entries in the queue pairs to be stored in locations that do not result in erroneously overwriting other entries in the queue pairs. Additionally, as part of these management functions, the one or more agents 194 may indicate one or more of the buffers 130A...130N that may be available to be reused.

[0040] As another example, one or more queue elements 116 may instruct this NVMe controller 112 to perform one or more read operations involving memory 150 and / or storage 156. Therefore, one or more commands 105 may also specify one or more locations (e.g., namespaces) in memory 150 and / or storage 156 from which NVMe controller 112 may read the data 199 as a result of executing one or more

[0041] Queue elements 116 reads.

[0042] According to some examples, in response to receiving one or more commands 105, the NW I / O device 108 may directly (e.g., according to an RDMA protocol (e.g., iWARP, IB, RoCE) and / or in a manner that bypasses and / or is independent of OSE 32) write one or more queue elements 116 to one or more submission queues 114 in the manner instructed by one or more commands 105. Therefore, by actually issuing one or more commands 105 to the NW I / O device 108, the NW I / O device 106 may write one or more queue elements 116 to one or more submission queues 114 and one or more buffers 130A.

[0043] In this example, one or more commands 105 may also include and / or specify one or more values ​​201 and one or more of the indicators 181 that may indicate one or more locations of one or more doorbells 170 to which one or more values ​​201 are to be written. In response to these one or more values ​​201 and these one or more of the indicators 181 in one or more commands 105, NW I / O device 108 may write one or more values ​​201 to the doorbell 170 directly (e.g., according to an RDMA protocol (e.g., iWARP, IB, RoCE) and / or in a manner that bypasses and / or is independent of the OSE 32) in the manner instructed by one or more commands 105. Writing one or more values ​​201 to the doorbell 170 can cause the doorbell 170 to ring.Therefore, by issuing one or more commands 105 to the NW I / O device 108, the NW I / O device 106 can actually ring the doorbell 170.

[0044] According to some examples, in response to the ringing of doorbell 170, NVMe controller 112 may return to a fully operational state (e.g., if NVMe controller 112 previously entered a reduced power state relative to that fully operational state) and read one or more queue items 116 written to one or more submission queues 114. NVMe controller 112 may then execute the one or more commands specified and / or embodied by one or more queue items 116. This may result in the NVMe controller 112 performing the one or more operations (e.g., one or more reads of the memory 150 and / or storage 156 to obtain the data 199) involving the memory 150 and / or storage 156, and storing the data 199 in one or more buffers (e.g., one or more buffers 130A).

[0045] In some examples, after completing one or more of these operations involving memory 150 and / or storage 156, the NVMe controller 112 may write one or more completion queue elements 129 to one or more completion queues 124. After completing one or more of these operations involving memory 150 and / or storage 156, the NVMe controller 112 may also write one or more values ​​to one or more doorbells 192 connected to the NW I / O device 108. This may ring one or more doorbells 192. In response to the ringing of one or more doorbells 192, the NW I / O device 108 may receive the queue elements 129 from the one or more completion queues 124 and forward or write one or more completion queue elements 190 to one or more completion queues 120 to complete the transmission of the data 199 (e.g.,via one or more RDMA write operations with NW I / O device 106) to one or more buffers 13 in client 10 (e.g., via one or more responses 197). Alternatively, handleability module 109 may receive queue items 129 from completion queues 124 and forward or write completion queue items 190 to completion queues 120 to facilitate data transfer 199 to buffers 13.

[0046] According to some examples, the command interface 110 may be asynchronous in that, for example, completion queue items may not be stored in one or more completion queues 120 in an order that corresponds to (1) the order in which command queue items are stored in the one or more submission queues 114, (2) the order in which such command queue items are forwarded by the NVMe controller 112 for execution and / or completion, and / or (3) the order in which the completion queue items 190 are stored in one or more completion queues 120 and / or provided to the NW I / O device 106 and / or to client 10.In operation, in the case of write commands issued by the client 10 and / or NW I / O device 106, the NW I / O device 106 and / or client 10 may appropriately reorder corresponding completion queue elements 190 received from the NW I / O device 108. However, in this embodiment, in the case of read commands, to allow a corresponding data read from memory 150 and / or storage 156 to be appropriately associated with corresponding completion queue elements 190 for transmission to the client 10 and / or NW I / O device 106, each completion queue element (e.g., completion queue element 190) resulting from completion indications placed into the completion queues 120 by the NW I / O device 108 shown in FIG. Fig. 2 include the elements illustrated.

[0047] As in Fig. 2, the completion queue element 200 (e.g., the completion queue element 190) may include one or more instruction parameters 304, one or more instruction queue identifiers 306, one or more instruction queue head position indicators 308, status information 310, one or more queue phase bits (P) 312, and / or one or more instruction identifiers 302. One or more instruction parameters 304 may be and / or indicate one or more instruction-specific parameters from the one or more queue elements 116 and / or the instructions 105 that may correspond to and / or be associated with the one or more completion queue elements 190. One or more command queue identifiers 306 may indicate and / or specify the one or more submission queues 114 to which the one or more queue elements 116 were written.One or more command queue head position indicators 308 may indicate the current position (e.g., in the one or more submission queues 114 identified by one or more command queue identifiers 306) at which the one or more queue elements 116 may be located. The status information 310 may indicate whether the one or more commands 105 and / or the one or more queue elements 116 were successfully executed by the NVMe controller 112. One or more phase bits 312 may indicate whether the one or more completion queue elements 190 represent the most recently added valid entry (e.g., to service) in one or more completion queue elements 120. One or more command identifiers 302 may indicate and / or be identical to one or more corresponding command identifiers in the corresponding one or more queue elements 116.The instruction identifiers 302 may allow one or more completion queue elements 190 to be properly associated with one or more corresponding queue elements 116 and / or with the corresponding data 199 read from the memory 150 and / or mass storage 156 as a result of the execution of those one or more corresponding queue elements 116.

[0048] In some examples, one or more command identifiers 302 may be selected so that they do not conflict with and / or are identical to any other command identifiers that may currently be used by any completion queue elements that have not yet been provided to client 10 and / or NW I / O device 106 by NW I / O device 108. The command identifiers used in system 100 may be pre-calculated and / or pre-generated and used as corresponding indices INDEX A ... INDEX N for corresponding entries ENTRY A ... ENTRY N in a table 250 that may be stored in memory 21. Each of the entries ENTRY A ... ENTRY N in the table 250 may store one or more corresponding pre-calculated and / or pre-generated command queue elements QE A ... QE N that may be connected to the NW I / O device 108. Each corresponding element QE A ...QE N may be associated with one or more corresponding buffers in one or more buffers 130A...130N. Each of the buffers in one or more buffers 130A...130N into which NVMe controller 112 may store data read from memory 150 and / or mass storage 156 may also be associated with one or more corresponding delivery identifiers used in system 100 and / or corresponding entries ENTRY A...ENTRY N.

[0049] The command queue elements QE A . . . QE N may be stored and / or maintained in table 250 by client 10 and / or one or more agents 194. If one or more buffers 130A ... 130N are statically allocated, table 250 may be static and, in terms of, for example, allocation properties, correspond to one or more buffers 13 that may be allocated in client 10.

[0050] For example, after NVMe controller 112 reads data 199 from memory 150 and / or storage 156, NVMe controller 112 may store data 199 in one or more buffers (e.g., one or more buffers 130A) that may be associated with one or more command identifiers 302, and may send an indication to NW I / O device 108 that an access command has completed, such as by ringing one or more doorbells 192. In response to NVMe controller 112 ringing one or more doorbells 192, NW I / O device 108 may determine the one or more most recently added valid completion queues in one or more completion queues 120 based on one or more queue phase bits 312.The NW I / O device 108 may use the one or more command identifiers 302 in one or more completion queue elements 190 to index into the table 250 and locate the one or more entries (e.g., one or more ENTRY A entries) and one or more command queue elements (e.g., one or more QEA queue elements) in the table 250 that may be associated with or identified by one or more command identifiers 302. The NW I / O device 108 may execute one or more commands that may be associated with and / or embodied by these one or more QEA command queue elements. For these examples, this may result in NW-I / O 108 reading one or more buffers 130A to obtain the data 199 and passing the data 199 and one or more completion queue elements 190 to the NW-I / O device 106 and / or client 10 (e.g.,via one or more responses 197). As a result, the data 199 and / or one or more completion queue elements 190 can be copied into one or more client buffers 13.

[0051] Alternatively, in some examples, the NW I / O device 108 may include a state machine (not shown). This state machine may be independent and / or separate from one or more submission queues 114 that may be connected to and / or used by the NW I / O device 108. This state machine may locate one or more command queue elements QE in the table 250 based on one or more command identifiers 302 and may copy the one or more queue elements QE to one or more corresponding submission queue elements 196A in one or more submission queues 126. The state machine may then signal the NW I / O device 108 to access and execute one or more submission queue elements 196A in one or more submission queues 126.

[0052] Further alternatively, without departing from these examples, prior to completing the one or more read operations involving memory 150 and / or storage 156, the NVMe controller 112 may locate and / or select one or more queue elements QEA in and / or from the table 250 based on one or more command identifiers 302. The NVMe controller 112 may then write to one or more completion queue elements 190 in one or more completion queues 120 and may write one or more queue elements QEA to one or more corresponding submission queue elements 196A in one or more submission queues 126. The NVMe controller 112 may then ring one or more doorbells 192.This may result in NW I / O device 108 accessing and executing one or more submission queue items 196A in one or more submission queues 126. For these examples, this may result in NW I / O device 108 reading one or more buffers 130A to obtain data 199 and transmitting data 199 and one or more completion queue items 190 to NW I / O device 106 and / or client 10 (e.g., via one or more responses 197). As a result, data 199 and / or one or more completion queue items 190 may be copied to one or more client buffers 13.

[0053] In this alternative example, firmware and / or one or more agents 194 executed by NW I / O device 108, NVMe controller 112, or manageability module 109 may maintain per-queue-pair context information to indicate one or more queue pairs used for the RDMA transactions. This context information may also include various pointers (e.g., to one or more arrays of the submission queue elements 196A...196N to move data from one or more buffers 130A...130N to one or more buffers 130A...130N and / or the head of one or more submission queues 126), one or more locations of one or more doorbells 192 and one or more values ​​to ring the one or more doorbells 192, and / or local copies of the head and / or pointers to the one or more submission queues 126. Various of these pointers (e.g.,the head and tail pointers) can be dynamically updated by firmware executed by NVMe controller 112.

[0054] Additionally or alternatively, without departing from these examples, NW I / O device 108, manageability module 109, and / or NVMe controller 112 may be included in the chipset (not shown) or in a board or device (not shown). Also additionally or alternatively, without departing from this embodiment, memory 150 and / or storage 156 may be located internally within server 20 or externally from server 20.

[0055] Although the foregoing description was made with reference to NW I / O device 108 as an RNIC and NVMe controller 112 as an NVMe-compliant host controller interface, the principles of this embodiment may be applied to situations where protocols other than and / or in addition to RDMA or NVMe may be employed, and / or where NVMe controller 112 may be involved in performing and / or facilitating operations that do not involve storage 150 (e.g., other and / or additional I / O and / or communication-related operations). Accordingly, without departing from the above examples, NW I / O device 108 and / or communications between client 10 and server 20 may employ protocols other than and / or in addition to RDMA.Furthermore, without departing from this embodiment, NW I / O device 108, manageability module 109, or NVMe controller 112 may be involved in performing and / or may facilitate the execution of such other and / or additional operations that may operate according to PCIe protocols or the NVMe interface. In these additional and / or alternative arrangements, hardware and / or firmware circuitry (not shown) may be included in circuitry 118 that allows writing to doorbells 170 and / or 192, for example, via one or more interrupt mechanisms (e.g., one or more Message Signaled Interrupts (MSI / MSI-X) and / or other mechanisms). This embodiment should be broadly considered to cover all such modifications, variations, and alternatives.

[0056] Therefore, in some examples, circuitry may be arranged to enable a first network I / O device in a client to access, via a second network I / O device in a server located remotely from the client, the command interface of another (e.g., memory and / or another / additional type of) controller of the server in a manner independent of an operating system environment in the server. The network I / O device in the client and the network I / O device in the server may be or include respective RDMA-capable network interface controllers (e.g., controllers that use and / or can communicate via RDMA). The command interface may include at least one (e.g., memory and / or another / additional type of) controller command queue.This accessing may include writing at least one queue element to the at least one submission queue to instruct the other controller to perform at least one operation (e.g., involving storage and / or one or more other and / or additional types of operations, such as other and / or additional input / output operations) associated with the other controller (e.g., an NVMe controller). The other controller may perform the at least one operation in response to at least one queue element. Many alternatives, variations, and modifications are possible. Some of these alternatives may include the use of a manageability module (e.g., manageability module 109) coupled between the NW I / O device and the NVMe controller at the server to facilitate access by the remote NW I / O device to the at least one command interface.

[0057] Therefore, in some examples, the one or more command interfaces 110 of the NVMe controller 112 in the server 20 may be accessed directly by the NVMe I / O device of the client 106 via one or more RDMA transactions in a manner that bypasses, is independent of, and / or does not involve the server's OSE 32 and / or the CPU 12. Advantageously, this may allow storage commands, data, and completion messages to be communicated between the client and the server much more quickly and efficiently, with reduced latency. Furthermore, in this embodiment, interactions between the NW I / O device 108 and NVMe controller 112 may be performed entirely or almost entirely by hardware (e.g., using peer-to-peer storage and doorbell writes) and also in a manner that bypasses, is independent of, and / or does not involve the OSE of the server 32 and / or CPU 12.Advantageously, this can allow these interactions to be performed much more quickly and efficiently, with reduced latency. Additionally, the above features of this embodiment can reduce server power consumption, heat dissipation, and the amount of bandwidth consumed by the OSE 32 and CPU 12.

[0058] Many other modifications are possible. For example, as previously mentioned in this embodiment, client 10 may comprise a plurality of clients. When RDMA is employed for communications between server 20 and clients 10, in this embodiment, clients 10 may advantageously dynamically share buffers 130A...130N as a common pool of buffers between or among clients 10 when performing their communications with server 20, NW I / O device 108, and / or NVMe controller 112. To allow this buffer sharing, NW I / O device 108 may be capable of manipulating, setting, and / or modifying buffer-specifying information that may be indicated in commands 105 provided to server 20 by clients 10 to use buffers 130A...130N.130N and / or other server resources to be shared among the clients 10 without, for example, resulting in contention-related problems.

[0059] For example, the one or more indicators 181 and / or STags indicated by the one or more indicators 181 may include corresponding information that the NW I / O device 108 may associate with one or more buffers and / or buffer pools in the buffers 130A...130N, instead of and / or in addition to one or more memory region handles. With this arrangement, the clients 10 may perform RDMA read operations using these indicators 181, and the NW I / O device 108 may perform write operations to the one or more buffers and / or buffer pools indicated by the corresponding information and / or indicators 181. In performing its operations, the NW I / O device 108 may appropriately adjust the actual commands and / or command queue elements provided to the NVMe controller 112, resulting in the correct buffers, etc.be written by the NVMe controller 112 when the NVMe controller 112 executes these commands and / or command queue items.

[0060] Alternatively or additionally, without departing from the above examples, the NW I / O device 108 may include and / or be coupled to a shared receive queue (not shown) for receiving, for example, the commands 105 from multiple clients 10. The NW I / O device 108 may be capable of substituting one or more suitable server buffer addresses, values, and / or other information in one or more portions (e.g., queue elements 116, values ​​201, indicators 181, and / or other information) of the received commands 105 to allow sharing of the structures in the one or more command interfaces 110 between or among multiple clients 10 without resulting in contention or other performance degradation.With this arrangement, clients may not be provided with and / or use one or more STags to the storage controller command queue and / or doorbell, and writing to these structures may be performed by the NW I / O device of server 108. Advantageously, this may allow multiple clients 10 that may be connected to and / or use the shared receive queue to use and / or share the same storage controller command queue, doorbell, and / or other structures.

[0061] For example, in the case of a write operation, one or more indicators 181, one or more values ​​201, and / or other information in one or more commands 105 may indicate one or more memory controller STags for the write operation (and related information) and / or one or more RDMA STags to one or more buffers to which one or more completion queue elements may be written. For these examples, based on the one or more received commands 105 and / or other information stored in the NW I / O device 108, the NW I / O device 108 may allocate one or more buffers in the buffers 130A...130N and select one or more locations in the submission queue 114 to which to post the data 199 to be written and one or more corresponding command queue items to be forwarded to the submission queue 126 associated with the NVMe controller 112. The NW I / O device 108 may post the data 199 and the one or more command queue items according to these selections and thereafter ring the doorbell 170. As posted by the NW I / O device 108, the one or more command queue items may indicate the one or more storage controller STags included in the one or more commands 105, command identifier 302, security context information (e.g.,to allow validation of the one or more storage controller STags) and / or one or more STags to the one or more buffers to which the data 199 was posted. After the NVMe controller 112 completes the requested one or more write operations and posts one or more completion queue items (e.g., to completion queue 124), the NVMe controller 112 may ring the doorbell 192. Based on information in the table 250, the NW I / O device 108 may generate and forward one or more appropriate responses 197 to the one or more clients that provided the receive commands 105 via forwarding the completion queue items from the completion queue 124 to the completion queue 120.

[0062] In the case of a read operation, generally analog information may be provided in command 105, and generally analog operations may be performed by the NW I / O device 108 and / or NVMe controller 112. However, in the case of a read operation, the data 199 read by the NVMe controller 112 may be stored by the NVMe controller 112 in one or more of the buffers 130A...130N specified by the NW I / O device 108 and may be read by the NW I / O device 108, rather than the other way around (e.g., as may be the case with a write operation). The NW I / O device 108 may send the read data 199 to the one or more clients that provided the received command 105 in one or more responses 197.In the foregoing arrangement, command 105 may be similar or identical to a command that may be used by a client to access client-local memory, at least from the perspective of one or more client-executed applications initiating that access. Advantageously, this may allow remote operations and / or RDMA transactions of the types described above to be substantially transparent to those one or more client-executed applications.

[0063] Therefore, in some examples, it may advantageously be possible for multiple clients to share the storage controller's command queue, doorbells, and / or the server's buffers and / or write to these structures (via the server's network I / O device) using an RDMA protocol without suffering from resource contention (and / or other disadvantages) that might otherwise occur. The server's network I / O device may be capable of modifying information associated with and / or included in the client's commands 105 to facilitate this sharing and / or the sharing of RDMA STag information between or among the clients.Advantageously, this may allow the RDMA protocol to be used for command communication and / or completion information between the server and the multiple clients with improved scalability, while reducing the memory requirements to implement such features, and without degradation in data transfer speed.

[0064] Fig. 3 illustrates a second exemplary system. As shown in Fig. 3, the second example includes a system 300. According to some examples, the system 300 may include multiple clients 305-1 through 305-n (where "n" represents any positive integer greater than 1) and a server 350. For these examples, each client may include an NW I / O device 310, a memory 320, a host processor (HP) 330, and a non-volatile memory (NVM) 340. Additionally, the server 350 may include an NW I / O device 352, an NVMe controller 354, and the storage device(s) 356. The clients 305-1 through 305-n or the server 350 may include additional elements or components described in Fig. 3 are not shown. Examples are not limited in this context.

[0065] In some examples, logic and / or features at clients 305-1 through 305-n, in cooperation with logic and / or features at server 350, may be capable of remotely booting using the storage device(s) 356 controlled by an NVMe controller 354 managed at server 350. Remote booting of any of clients 305-1 through 305-n may be initiated at or near at least one of clients 305-1 through 305-n (e.g., by toggling a power button), at or near server 350, or at a centralized control point (not shown) that may manage the operations of system 300 (e.g., initiated by an operator). Remote booting can also be initiated due to power cycles (intentional or unintentional) or due to a client coming online on a given network.

[0066] According to some examples, as part of pre-boot activities, NW I / O devices 310-1 through 310-n and 352 may be capable of exchanging data and / or commands over respective NW communication links 360-1, 360-2, and 360-3 coupled to network 360, according to one or more protocols that may be compatible with one or more RDMA protocols, such as iWARP, Infiniband, Ethernet, TCP / IP, or RoCE. For example, logic and / or features may be executed by circuitry 312-1 for NW I / O device 310-1 at client 305-1 to connect to a remotely located server 350 over NW communication link 360-1 coupled to network 360 using at least one of these RDMA protocols.

[0067] In some examples, the logic and / or features for execution by circuitry 312-1 may receive one or more parameters to enable network I / O device 310-1 to connect to server 350 via network communication link 360-1. According to some examples, the one or more parameters may be received or obtained from non-volatile memory (NVM) 314-1 managed at network I / O device 310-1. The one or more parameters may include information associated with establishing a network connection to server 350. For example, the one or more parameters may include, among other things, an Internet Protocol (IP) address for the server 350, authentication information to authenticate the NW I / O device 310-1 and / or client 305-1 with the server 350, or an identifier (e.g., node identification) for the client 305-1.The one or more parameters may also include network identification information for the network communication link 360-1 to facilitate the exchange of data and / or commands through the network 360 via a network connection to the server 350. The network information may include, among other things, a LAN identifier or a VLAN identifier.

[0068] According to some examples, the logic and / or features for execution by circuitry 312-1 may receive the one or more parameters from a combination of sources. For example, a first portion of the one or more parameters may be received from NVM 314-1, and a second portion may be received from server 350 (e.g., managed / stored in NVM 353). The first portion may include, but is not limited to, an identifier for client 305-1 or authentication information to authenticate client 305-1 with server 350. The second portion may include information used by client 305-1 to locate and communicate with server 350, such as an IP address for server 350 or network identification information to include, among other things, a LAN or VLAN identifier.

[0069] In some examples, the logic and / or features for execution by circuitry 321-1 may also establish a control path to NVMe controller 354 managed at server 350. For these examples, an RDMA protocol such as iWARP, Infiniband, Ethernet, TCP / IP, or RoCE may be used to establish the control path to NVMe controller 354 in a manner similar to that described above for establishing a control path to NVMe controller 112 for Fig. 1. Once the control path is established, the logic and / or features may then be capable of determining one or more properties for the storage device(s) 356. The one or more properties may include, but are not limited to, storage device properties such as the number of storage devices, type(s) of storage for each storage device, available storage capacity or access rights (e.g., read-only memory (ROM) or read / write (R / W) memory), directory information, or share names.

[0070] According to some examples, logic and / or features for execution by circuitry 321-1 at NW I / O device 310-1 may be capable of registering the one or more determined properties with a system BIOS 341-1. As shown in Fig. 3, for some examples, system BIOS 341-1 may be managed in NVM 340-1 and executed by client 305-1 circuits such as HP 330-1. As further described below, the registered one or more detected properties may be used to facilitate remote booting of client 305-1 via remote access to storage device(s) 356.

[0071] Although not shown in Fig. 3, in some examples, instead of registering the one or more determined properties with a system BIOS, logic and / or features for execution by circuitry 321-1 at NW I / O device 310-1 may also be capable of operating in cooperation with other types of boot-related / startup firmware that may be stored in NVM 340-1. For example, the firmware may be for a Unified Extensible Firmware Interface (UEFI) or an Extensive Firmware Interface (EFI). This UEFI or EFI firmware may perform similar functions as described in this disclosure for BIOS 341-1.

[0072] In some examples, logic and / or features for execution by circuitry 321-1 at NW I / O device 310-1 may be capable of receiving an RDMA STag from NVMe controller 354 (e.g., via the established control path). For these examples, the RDMA STag may represent an assigned portion of storage device(s) 356 that may be accessible using the RDMA STag. For example, the received RDMA STag may represent access to one of assigned portions 355-1, 355-2, or 355-3. Additionally, for these examples, a given assigned portion represented by the received RDMA STag may store an OS kernel and one or more device drivers that may be used to remotely boot client 305-1.

[0073] According to some examples, logic and / or features for execution by circuitry 321-1 may store a received RDMA STag representing allocated portions in non-volatile memory such as NVM 314-1 maintained at NW I / O device 310. For these examples, the RDMA STag may be accessible to system BIOS 341-1 to enable system BIOS 341-1 to use the RDMA STag to access storage device(s) 356 and load OS kernel 322-1 and device driver(s) 324-1 into memory 320-1 to remotely boot client 305-1.

[0074] In some examples, allocated portions of the storage device(s) 356 accessible using the received RDMA STag may be jointly accessible by other clients connected to the server 350 via respective separate network communication links. For these examples, a same allocated portion of the storage device(s) 356 may be shared between the clients 305-1 through 305-n to consolidate storage resources. For example, an RDMA STag representing allocated portion 355-1 may enable read-only access to allocated portion 355-1. Shared access to the same OS kernel and device drivers may enable unified remote booting of the clients 305-1 through 305-n.

[0075] In some other examples, the received RDMA STag may enable both shared read-only access to allocated portions of the storage device(s) 356 and private read-write access to allocated portions of the storage device(s) 356. For these examples, the shared allocated portions of the storage device(s) 356 may be system-wide related information used to remotely boot all clients of the system 300. Meanwhile, the private read-write access may enable client-specific information to be used to remotely boot a given client.For example, an RDMA STag may represent allocated portion 355-2, and logic and / or features at server 350 may be capable of determining whether the RDMA STag also enables a given client to access shared (read-only 355-3A) and private allocated portions (R / W 335-3B1 or 335-3Bn). According to some examples, if the given client is client 305-1, the logic and / or features at server 350 may determine that an RDMA STag used by client 305-1 to access allocated portion 355-2 enables access to both read-only 355-3A and R / W 355-3B1. If the given client is client 305-n, the logic and / or features at server 350 may also determine that an RDMA STag used by client 305-n to access assigned portion 355-2 allows access to both read-only 355-3A and R / W 355-3Bn.Therefore, for these examples, logic and / or features at the given client may be unaware of the first and second portions accessible via the RDMA STag. In some other examples, separate RDMA STags (one for the read-only 355-3A and one for the R / W 355B) may be used to provide access to the allocated portion 355-2. For these other examples, logic and / or features at the given client may determine which allocated portion must be accessed to remotely boot the given client device.

[0076] According to some examples, a client such as client 305-1 may use the RDMA STag, which provides access to the allocated portion 355-2, to enable the system BIOS 341-1 to first load 320-1 from the read-only 355-3A OS kernel 322-1 and the device driver(s) 324-1 into memory. For these examples, the OS kernel 322-1 and the device driver(s) 324-1 may then use the RDMA STag to obtain information from the R / W 355-3B 1 specific to and / or allocated to the configuration of client 305-1. For some examples, the information specific to and / or assigned to the configuration of client 305-1 may include, but is not limited to, configuring client 305-1 to operate as a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server hosting one or more hosts.

[0077] Fig. 4 illustrates a remote boot scheme 400. As shown in Fig. 4, remote boot 400 includes a pre-boot and OS runtime portion. According to some examples, remote boot scheme 400 may be implemented by elements of client 305-1. For example, as shown in Fig. 4, the circuits 312 of the NW I / O device 310 may implement logic and / or features to use the NW communication link 360-1 to communicate with a remote server such as server 350 through the network 360 as mentioned above for the Fig. 3. In addition, the system BIOS 341-1 can be used to transition from pre-boot to OS runtime operations, as described in more detail below.

[0078] According to some examples, a remote NVMe boot read-only memory (ROM) 410 may be located, loaded, and then executed by the circuitry 312 at network I / O device 310. For these examples, remote NVMe boot ROM 410 may be originally managed / stored in NVM 314 and, when loaded, may include a pre-boot network I / O driver, remote NVMe transport, and BIOS disk services as shown in Fig. 4. For example, using the pre-boot network I / O driver and the remote NVMe transport, logic and / or features executed by circuitry 312 may connect to a remote server, establish a control path to an NVMe controller managed at the server, and determine one or more properties of the storage device(s) controlled by the NVMe controller. Additionally, BIOS disk services may be used by the logic and / or features to communicate or register the one or more determined properties of the storage device(s) with the system BIOS 341-1.

[0079] In some examples, BIOS disk services included as part of the remote NVMe boot ROM 410 may include the loading of BIOS interrupt call 13hex (INT 13h) services for use by the system BIOS 314-1 to facilitate the initial loading of an OS kernel and device driver(s) from the storage device(s) controlled by the NVMe controller managed at the remote server. For these examples, the OS kernel 322-1 and device driver(s) 324-1 may be loaded (e.g., into memory 320-1), and OS runtime operation may begin. As part of the OS runtime, a remote NVMe storage stack 420 may be implemented by the OS kernel 322-1. As shown in Fig. 4, an NW I / O device driver 430 may also be configured / implemented from the number of device drivers 324-1 to maintain a remote NVMe connection using RDMA protocols to communicate with the NVMe controller at the remote server.

[0080] According to some examples, as highlighted by the arrow between NVM 314 and NW I / O device driver 430, information stored / managed at NVM 314 may be provided to facilitate maintaining the remote NVMe connection. For example, logic and / or features of NW I / O device 310 may be capable of supporting BIOS Enhanced Disk Driver (EDD) services. Support for the BIOS EDD may enable the logic and / or features of NW I / O device 310 to enable a seamless transition from pre-boot to OS runtime, while exchanging / making available parameters, properties, and / or RDMA STags obtained during pre-boot to the OS kernel 322-1 or the device driver(s) 324-1 during OS runtime.

[0081] Fig. 5 illustrates an exemplary block diagram of a first device. As shown in Fig. 5, the first device includes device 500. Although device 500 shown in Fig. 5 has a limited number of elements in a particular topology, it is conceivable that the device 500 may include more or fewer elements in alternative topologies as desired for a given implementation.

[0082] The apparatus 500 may be supported by circuitry 520 managed at a network I / O device coupled to a client or host device. The circuitry 520 may be arranged to execute one or more software- or firmware-implemented components or modules 522-a. It should be noted that "a," "b," "c," and similar identifiers as used herein are intended as variables representing any positive integer. For example, if an implementation specifies a value for a = 7, then a complete set of software or firmware for modules 522-a may include modules 522-1, 522-2, 522-3, 522-4, 522-5, 522-6, or 522-7. The examples presented are not limited in this context, and the different variables used throughout may represent the same or different integer values.

[0083] According to some examples, circuitry 520 may include a processor or processor circuitry. The processor or processor circuitry may be any of various commercially available processors, including, without limitation, AMD® Athlon®, Duron®, and Opteron® processors; ARM® application, embedded, and secure processors; IBM® and Motorola® DragonBall® and PowerPC® processors; IBM and Sony® Cell processors; Intel® Atom®, Celeron®, Core (2) Duo®, Core i3, Core i5, Core i7, Itanium®, Pentium®, Xeon®, Xeon Phi®, and XScale® processors; and similar processors. According to some examples, circuitry 520 may also be an application-specific integrated circuit (ASIC), and at least some modules 522-a may be implemented as hardware elements of the ASIC.

[0084] According to some examples, device 500 may include a parameter module 522-1 for execution by circuitry 520. Parameter module 522-1 may be capable of receiving parameters 505 to enable a network I / O device with device 500 to connect to a remote server via the network communication link. In some examples, the one or more parameters may be received from non-volatile memory maintained at the network I / O device, or at least a portion of the parameters may be received from the remote server.

[0085] In some examples, device 500 may also include a connection module 522-2 for execution by circuitry 520. Connection module 522-2 may be capable of connecting to the remote server using the one or more parameters received by parameter module 522-1.

[0086] The connection module 522-2 may be capable of at least temporarily storing the parameter information 524-a (e.g., in a data structure such as a lookup table (LUT)) to facilitate connecting to the remote server. The parameter information 524-a may include information used by the connection module 522-2 to locate the remote server (e.g., an IP address), identify the client or host device to the remote server, authenticate the client or host device to the remote server, or network identification information such as a LAN identifier or a VLAN identifier.

[0087] In some examples, device 500 may also include a path module 522-3 for execution by circuitry 520. Path module 522-3 may be capable of establishing a control path of an NVMe controller managed at the remote server using one or more RDMA protocols. For these examples, path module 522-3 may be capable of at least temporarily storing RDMA protocol information 525-b (e.g., in a LUT) to establish the control path. The one or more RDMA protocols may include, among others, iWARP, Infiniband, or RoCE protocols.

[0088] According to some examples, device 500 may also include a property module 522-4 for execution by circuitry 520. Property module 522-4 may be capable of determining one or more properties 510 for a storage device or devices controlled by the NVMe controller at the remote server. For these examples, property module 522-4 may at least temporarily store determined property information 526-c from properties 510 in a data structure such as a LUT or in non-volatile memory managed at the NVMe I / O device including device 500. The one or more determined properties may include, but are not limited to, a number of storage devices controlled by the NVMe controller, properties of the storage device(s), such as types of storage, available capacity, or access rights, such as ROM or R / W storage.

[0089] In some examples, device 500 may also include a registration module 522-5 for execution by circuitry 520. Registration module 522-5 may be capable of registering the storage device(s) and the one or more determined properties with a system BIOS for the host or client device coupled to the NW I / O device having device 500. For these examples, registration module 522-5 may at least temporarily store storage device / property information 527-d in a data structure, such as non-volatile memory, maintained at the NW I / O device. Additionally, registration module 522-5 may provide registered information 535 to the system BIOS as part of registering the storage device(s) and the one or more determined properties.The registered information 535 may include information for the system BIOS to locate the memory / device property information 527-d, such as a pointer to a memory address of a non-volatile memory managed at the NW I / O device.

[0090] According to some examples, device 500 may also include a receive module 522-6 for execution by circuitry 520. Receive module 522-6 may be capable of receiving RDMA STag 515 from the NVMe controller at the remote server. For these examples, RDMA STag 515 may include an RDMA STag representing allocated portions of a storage device controlled by the NVMe controller accessible using the RDMA STag. Additionally, for these examples, the allocated portion(s) may store an OS kernel and one or more device drivers to be used for remotely booting the host or client device coupled to the NVMe I / O device comprising device 500.

[0091] In some examples, device 500 may also include a storage module 522-7 for execution by circuitry 520. Storage module 522-7 may be capable of storing the received RDMA STag encapsulated in RDMA STag 515 in non-volatile memory, as the stored RDMA STag 530 at the NW I / O device includes device 500. For these examples, stored RDMA STag 530 may be accessible to the system BIOS for the host or client device coupled to the NW I / O device. According to some examples, the system BIOS may use the RDMA STag encapsulated in stored RDMA STag 530 to access the storage device and load the OS kernel and one or more device drivers to remotely boot the host or client device.

[0092] Included herein is a set of logic flows representing exemplary methodologies for implementing new aspects of the disclosed architecture. While, for ease of explanation, the one or more methodologies shown herein are shown and described as a series of acts, those skilled in the art will understand that the methodologies are not limited by the order of the acts. Accordingly, some acts may occur in a different order and / or concurrently with other acts shown and described herein. One skilled in the art will understand that a methodology could alternatively be represented as a series of interrelated states or events, such as in a statechart diagram. Furthermore, not all acts illustrated in a methodology may be required for a new implementation.

[0093] A logic flow may be implemented in software, firmware, and / or hardware. In software and firmware embodiments, a logic flow may be implemented by computer-executable instructions stored on at least one non-transitory computer-readable medium or machine-readable medium, such as optical, magnetic, or semiconductor storage. The embodiments are not limited in this context.

[0094] Fig. 6 illustrates an example of a first logic flow. As shown in Fig. 6, the first logic flow includes logic flow 600. Logic flow 600 may represent some or all of the operations performed by one or more logic, features, or devices, such as device 500, described herein. In particular, logic flow 600 may be implemented by parameter module 522-1, connection module 522-2, path module 522-3, receive module 522-6, or storage module 522-7.

[0095] According to some examples, at block 602, logic flow 600 may receive one or more parameters from a network I / O device coupled to a host device to enable the network I / O device to connect to a remote server via a network communication link. For example, parameters 505 including the one or more parameters may be received from parameter module 522-1 included in an apparatus 500 for a network I / O device coupled to the host device.

[0096] In some examples, at block 604, logic flow 600 may connect to the remote server using the one or more parameters. For example, connection module 522-2 may use parameters included in parameters 505 to connect to the remote server. The one or more parameters may include information to facilitate connection over the network communication link, an IP address for the remote server, identifier information for the host device, or authentication information to authenticate the host device to the remote server.

[0097] According to some examples, at block 606, logic flow 600 may then establish a control path to an NVMe controller managed at the remote server using an RDMA protocol. For example, path module 522-3 may use RDMA protocol information 524-b, which includes information to specify a control path using protocols such as iWARP, Infiniband, or RoCE.

[0098] In some examples, at block 608, logic flow 600 may receive an RDMA STag from the NVMe controller representing an allocated portion of the storage device that may be accessed using the RDMA STag. For these examples, the allocated portion may store an OS kernel and one or more device drivers. For example, receive module 522-6 may receive RDMA STag 515, which includes the RDMA STag representing the allocated portion of the storage device.

[0099] According to some examples, at block 610, logic flow 600 may then store the RDMA STag in non-volatile memory at the NW I / O device, which may be accessible to a system BIOS for the host device, to enable the system BIOS to use the RDMA STag to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the host device. For example, storage module 522-7 may store the received RDMA STag in stored RDMA STag 530 maintained in non-volatile memory at the NW I / O devices including apparatus 500. The system BIOS may then use the RDMA STag in stored RDMA STag 530 to load the OS kernel and the one or more device drivers to remotely boot the host device.

[0100] Fig. 7 illustrates an example of a first storage medium. As shown in Fig. 7, the first storage medium includes storage medium 700. Storage medium 700 may comprise an article of manufacture. In some examples, storage medium 700 may include any non-transitory computer-readable medium or any machine-readable medium, such as optical, magnetic, or semiconductor memory. Storage medium 700 may store various types of computer-executable instructions, such as instructions to implement logic flow 600. Examples of a computer-readable medium may include any physical medium capable of storing electronic data, for example, volatile memory or non-volatile memory, removable memory or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and so on.Examples of computer-executable instructions can include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, and the like. The examples are not limited in this context.

[0101] Fig. Figure 8 illustrates an example NW I / O device 800. In some examples, as shown in Fig. 8, the NW I / O device 800 may include a processing component 840, other platform components, or a communication interface 860. According to some examples, the NW I / O device 800 may be implemented in an NW I / O device coupled to a host or client device as mentioned above.

[0102] According to some examples, processing component 840 may perform processing operations or logic for device 500 and / or storage medium 700. Processing component 840 may include various hardware elements, software elements, or a combination of both. Examples of hardware elements may include devices, logic devices, components, processors, microprocessors, circuits, processor circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so on), integrated circuits, application-specific integrated circuits (ASICs), programmable logic units (PLDs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), memory units, logic gates, registers, a semiconductor device, chips, microchips, chipsets, and so on.Examples of software elements may include software components, programs, applications, computer programs, application programs, device drivers, system programs, software development programs, machine programs, operating system software, middleware, firmware, software modules, utilities, subroutines, functions, methods, procedures, software interfaces, application programming interfaces (APIs), instruction sets, computer code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.However, the determination of whether an example is implemented using hardware elements and / or software elements may vary in accordance with any number of factors, such as desired computational speed, power levels, thermal tolerances, processing cycle budget, input data speeds, output data speeds, memory resources, data bus speeds, and other design or performance constraints as desired for a particular implementation.

[0103] In some examples, other platform components 850 may include common computing elements, such as one or more processors, multi-core processors, coprocessors, memory units, chipsets, controllers, peripherals, interfaces, oscillators, timing devices, and so forth. Examples of memory units may include, without limitation, various types of computer-readable and machine-readable storage media in the form of one or more higher-speed memory units, such as ROM, RAM, DRAM, DDRAM, SDRAM, SRAM, PROM, EPROM, EEPROM, flash memory, or any other type of storage medium suitable for storing information.

[0104] In some examples, communication interface 860 may include logic and / or features to support a communication interface. For these examples, communication interface 860 may include one or more communication interfaces operating according to various communication protocols or standards to communicate over direct or network communication links. Direct communications may occur via the use of communication protocols or standards described in one or more industry standards (including successors and variants) such as those associated with the PCIe specification, the NVMe specification, the RDMA protocol specification, the IEEE 802-2-2008 specification, RFC 791, or RFC 793.

[0105] The components and features of NW I / O device 800 may be implemented using any combination of discrete circuits, application-specific integrated circuits (ASICs), logic gates, and / or single-chip architectures. Furthermore, the features of NW I / O device 800 may be implemented using microcontrollers, programmable logic arrays, and / or microprocessors, or any combination of the foregoing, where appropriate. It should be noted that hardware, firmware, and / or software elements may be collectively or individually referred to herein as "logic" or "circuitry."

[0106] It is understood that the exemplary NW I / O device 800 shown in the block diagram of Fig. 8 may represent a functionally descriptive example of many potential implementations. Accordingly, any partitioning, omission, or inclusion of block functions depicted in the accompanying drawings does not imply that the hardware components, circuits, software, and / or elements for implementing those functions are necessarily partitioned, omitted, or included in the embodiments.

[0107] Fig. 9 illustrates an exemplary block diagram of a second device. As shown in Fig. 9, the second device includes device 900. Although device 900 shown in Fig. 9 has a limited number of elements in a particular topology, it is conceivable that the device 900 may include more or fewer elements in alternative topologies as desired for a given implementation.

[0108] Apparatus 900 may be supported by circuitry 920 managed at a network I / O device located at or coupled to a server, which may be remote from one or more client or host devices. Circuitry 920 may be arranged to execute one or more software- or firmware-implemented components or modules 922-a. It should be noted that "a," "b," "c," and similar identifiers as used herein are intended to be variables representing any positive integer. For example, if an implementation specifies a value of a = 5, then a complete set of software or firmware for modules 922-a may include modules 922-1, 922-2, 922-3, 922-4, or 922-5.The examples presented are not limited in this context and the different variables used throughout may represent the same or different integer values.

[0109] According to some examples, circuitry 920 may include a processor or processor circuitry. The processor or processor circuitry may be any of various commercially available processors, including, without limitation, AMD® Athlon®, Duron®, and Opteron® processors; ARM® application, embedded, and secure processors; IBM® and Motorola® DragonBall® and PowerPC® processors; IBM and Sony® Cell processors; Intel® Atom®, Celeron®, Core (2) Duo®, Core i3, Core i5, Core i7, Itanium®, Pentium®, Xeon®, Xeon Phi®, and XScale® processors; and similar processors. According to some examples, circuitry 920 may also be an application-specific integrated circuit (ASIC), and at least some modules 922-a may be implemented as hardware elements of the ASIC.

[0110] According to some examples, device 900 may include a parameter module 922-1 for execution by circuitry 920. Parameter module 922-1 may be capable of receiving parameters 905 to enable a network I / O device with device 900 to connect to a remote client via a network communication link. In some examples, the one or more parameters may be received from non-volatile memory maintained at the network I / O device, or at least a portion of the parameters may be received from the remote client (e.g., authentication information).

[0111] In some examples, apparatus 900 may also include a connection module 922-2 for execution by circuitry 920. Connection module 922-2 may be capable of connecting to the remote client using the one or more parameters included in parameters 905 received by parameter module 922-1. Connection module 922-2 may be capable of at least temporarily storing information from parameters 905 in parameter information 924-a (e.g., in a data structure such as a LUT) to facilitate connecting to the remote client. The parameter information 924-a may include information used by the connection module 922-2 to locate the remote client (e.g., an IP address), identify the remote client, authenticate the remote client, or network identification information such as a LAN identifier or a VLAN identifier.

[0112] In some examples, apparatus 900 may also include a path module 922-3 for execution by circuitry 920. Path module 922-3 may be capable of establishing a control path between the remote client and an NVMe controller managed at the server using one or more RDMA protocols. For these examples, path module 922-3 may be capable of at least temporarily storing RDMA protocol information 925-b (e.g., in a LUT) to establish the control path. The one or more RDMA protocols may include, among others, iWARP, Infiniband, or RoCE protocols.

[0113] According to some examples, device 900 may also include a forwarding module 922-4 for execution by circuitry 920. Forwarding module 922-4 may be capable of forwarding one or more properties 910 for a storage device or devices controlled by the NVMe controller at the server to the remote client. The one or more forwarded properties may include, but are not limited to, a number of storage devices controlled by the NVMe controller, properties of the storage device(s) such as types of storage, available capacity, or access rights, such as ROM or R / W storage.The forwarding module 922-4 may also be capable of forwarding an RDMA STag 915 generated by the NVMe controller that represents an allocated portion of a given storage device or storage devices that may be accessible using one or more RDMA STags included in the RDMA STag 915. For these examples, the allocated portion may include an OS kernel and one or more device drivers.

[0114] According to some examples, device 900 may also include a receive module 922-5 for execution by circuitry 920. Receive module 922-5 may be capable of receiving the one or more RDMA STags included in RDMA STag 915 forwarded to the remote client by forwarding module 922-4. For these examples, the received RDMA STag 915 may then be forwarded to the NVMe controller to enable the remote client to access the one or more storage devices controlled by the NVMe controller and load the OS kernel and one or more device drivers to remotely boot the remote client.

[0115] Fig. 10 illustrates an example of a first logic flow. As shown in Fig. 10, the second logic flow includes logic flow 1000. Logic flow 1000 may be some or all of the operations performed by one or more logic, features, or devices such as device 900 described herein. In particular, logic flow 1000 may be implemented by parameter module 922-1, connection module 922-2, path module 922-3, forwarding module 922-4, or receiving module 922-5.

[0116] According to some examples, at block 1002, logic flow 1000 may receive one or more parameters associated with connecting to a remote client via a network communication link. For example, parameters 905 may be received by parameter module 922-1 included in an apparatus 900 for a network I / O device coupled to a server that may be remote from the client.

[0117] In some examples, at block 1004, logic flow 1000 may connect to the remote client over the network communication link using the one or more parameters. For example, connection module 922-2 may use parameters included in parameters 905 to connect to the remote client. The one or more parameters may include information to facilitate the connection over the network communication link, an IP address for the remote client, identifier information for the server, or authentication information to enable the server to authenticate the remote client.

[0118] According to some examples, at block 1006, logic flow 1000 may then establish a control path between the remote client and an NVMe controller managed at the server. For these examples, the control path may be established using an RDMA protocol. For example, path module 922-3 may use RDMA protocol information 924-b, which includes information to facilitate establishing a control path between the NVMe controller and the remote client using protocols such as iWARP, Infiniband, or RoCE.

[0119] In some examples, at block 1008, logic flow 1000 may forward one or more properties for a storage device controlled by the NVMe controller to the remote client and send an RDMA STag generated by the NVMe controller representing an allocated portion of the storage device accessible using the RDMA STag. For these examples, the allocated portion may include an OS kernel and one or more device drivers. For example, forwarding module 922-4 may forward properties 910 and RDMA STag 915 to the remote client.

[0120] According to some examples, at block 1010, logic flow 10000 may receive the RDMA STag from the remote client and may forward the RDMA STag to the NVMe controller to enable the remote client to access the storage device and load the OS kernel and one or more device drivers to remotely boot the remote client. For example, receive module 922-5 may receive the RDMA STag 915 and then forward the RDMA STag 915 to the NVMe controller to enable the remote client to access the storage device controlled by the NVMe controller.

[0121] Fig. 11 illustrates an example of a second storage medium. As shown in Fig. 11, the second storage medium includes storage medium 1100. Storage medium 1100 may comprise an article of manufacture. In some examples, storage medium 1100 may include any non-transitory computer-readable medium or any machine-readable medium, such as optical, magnetic, or semiconductor memory. Storage medium 1100 may store various types of computer-executable instructions, such as instructions to implement logic flow 1000. Examples of a computer-readable medium may include any physical medium capable of storing electronic data, for example, volatile memory or non-volatile memory, removable memory or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and so on.Examples of computer-executable instructions can include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, and the like. The examples are not limited in this context.

[0122] Fig. 12 illustrates an example NW I / O device 1200. In some examples, as shown in Fig. 12, the NW I / O device 1200 may include a processing component 1240, other platform components, or a communications interface 1260. According to some examples, the NW I / O device 1200 may be implemented in an NW I / O device coupled to a server that can couple to a remote client or host device as mentioned above.

[0123] According to some examples, processing component 1240 may perform processing operations or logic for device 900 and / or storage medium 1100. Processing component 1240 may include various hardware elements, software elements, or a combination of both.

[0124] In some examples, other platform components 1250 may include common computing elements such as one or more processors, multi-core processors, coprocessors, memory units, chipsets, controllers, peripherals, interfaces, oscillators, timing devices, and so forth.

[0125] In some examples, the communication interface 1260 may include logic and / or features to support a communication interface. For these examples, the communication interface 1260 may include one or more communication interfaces operating according to various communication protocols or standards to communicate over direct or network communication links. Direct communications may occur via the use of communication protocols or standards described in one or more industry standards (including successors and variants) such as those associated with the PCIe specification, the NVMe specification, the RDMA protocol specification, the IEEE 802-2-2008 specification, RFC 791, or RFC 793.

[0126] The components and features of NW I / O device 1200 may be implemented using any combination of discrete circuits, application-specific integrated circuits (ASICs), logic gates, and / or single-chip architectures. Furthermore, the features of NW I / O device 1200 may be implemented using microcontrollers, programmable logic arrays, and / or microprocessors, or any combination of the foregoing, where appropriate. It should be noted that hardware, firmware, and / or software elements may be collectively or individually referred to herein as "logic" or "circuitry."

[0127] It is understood that the exemplary NW I / O device 1200 shown in the block diagram of Fig.12 may represent a functionally descriptive example of many potential implementations. Accordingly, any partitioning, omission, or inclusion of block functions depicted in the accompanying drawings does not imply that the hardware components, circuits, software, and / or elements for implementing those functions are necessarily partitioned, omitted, or included in the embodiments.

[0128] One or more aspects of at least one embodiment may be implemented by representative instructions stored on at least one machine-readable medium that represent various logic within the processor that, when read by a machine, computing device, or system, cause the machine, computing device, or system to generate logic for performing the techniques described herein. Such representations, also known as "IP cores," may be stored on tangible machine-readable medium and delivered to various customers or manufacturing facilities, where they are loaded into manufacturing machines that manufacture the logic or processor.

[0129] Various examples can be implemented using hardware elements, software elements, or a combination of both. In some examples, hardware elements can include devices, logic devices, components, processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so on), integrated circuits, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), memory units, logic gates, registers, a semiconductor device, chips, microchips, chipsets, and so on.In some examples, software elements may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, utilities, subroutines, functions, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computer code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.However, the determination of whether an example is implemented using hardware elements and / or software elements may vary in accordance with any number of factors, such as desired computational speed, power levels, thermal tolerances, processing cycle budget, input data speeds, output data speeds, memory resources, data bus speeds, and other design or performance constraints as desired for a particular implementation.

[0130] Some examples may include an article of manufacture or at least one computer-readable medium. A computer-readable medium may include a non-volatile storage medium for storing logic. In some examples, the non-volatile storage medium may include one or more types of computer-readable storage media capable of storing electronic data, for example, volatile memory or non-volatile memory, removable memory or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and so on.In some examples, logic may include various software elements such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, utilities, subroutines, functions, methods, procedures, software interfaces, programming interfaces, instruction sets, computer code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.

[0131] According to some examples, a computer-readable medium may include a non-transitory storage medium for storing or maintaining instructions that, when executed by a machine, computing device, or system, cause the machine, computing device, or system to perform methods and / or operations according to the described examples. The instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. The instructions may be implemented according to a predefined computer language, manner, or syntax to instruct a machine, computing device, or system to perform a particular function. The instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0132] Some examples may be described using the phrase "in an example" or "an example," along with their derivatives. These terms mean that a particular feature, structure, or characteristic described in connection with the example is included in at least one example. The use of the phrase "in an example" in various places in the description does not necessarily always refer to the same example.

[0133] Some examples can be described using the terms "coupled" and "connected," along with their derivatives. However, these terms are not necessarily synonymous. For example, descriptions using the terms "connected" and / or "coupled" may indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" can also mean that two or more elements are not in direct contact with each other, but still work or interact with each other.

[0134] In some examples, an example first device for a network I / O device may include circuitry. The example first device may also include a parameter module, for execution by the circuitry, to receive one or more parameters to enable the network I / O device to connect to a remote server via a network communication link. The example first device may also include a connection module, for execution by the circuitry to connect to the remote server using the one or more parameters. The example first device may also include a path module, for execution by the circuitry to establish a control path to an NVMe controller managed at the remote server using an RDMA protocol.The example first device may also include a receive module for execution by the circuitry to receive an RDMA STag from the NVMe controller to access an OS core and one or more device drivers.

[0135] In some examples of the example first device, the RDMA STag may represent a dedicated portion of a storage device controlled by the NVMe controller that is accessible using the RDMA STag to access the OS kernel and the one or more device drivers.

[0136] According to some examples, the example first device may also include a storage module for execution by the circuitry to store the RDMA STag in non-volatile memory at the network I / O device that is accessible to a system BIOS for a host device coupled to the network I / O device, to enable the system BIOS to use the RDMA STag to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the host device.

[0137] In some examples for the example first device, the connection module for the example first device may relinquish control of the connection to the remote server via the network communication link to the OS kernel or the one or more device drivers. Additionally, the storage module may cause the RDMA STag to be stored in non-volatile memory at the network I / O device to be accessible to the OS kernel or the one or more device drivers to facilitate access to the storage device by the OS kernel or the one or more device drivers using the RDMA STag.

[0138] According to some examples for the example first device, the allocated portion of the storage device accessible using the RDMA STag may be jointly accessible by one or more other host devices connected to the remote server via respective separate network communication links.

[0139] In some examples, the allocated portion of the storage device accessible using the RDMA STag may include a first portion and a second portion. For these examples, the first portion may be read-only and shared by one or more other host devices connected to the remote server via respective separate network communication links. The second portion may be read / write and accessible only to the host device. According to some examples, the second portion may include information allocated to the host device and used to load the OS kernel or the one or more device drivers for execution by circuitry at the host device.

[0140] According to some examples of the example first apparatus, the information assigned to the host device may include information to configure the host device as one or more of a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server.

[0141] In some examples, the example first device may also include a properties module for execution by the circuitry to determine one or more properties for the storage device controlled by the NVMe controller. The example first device may also include a registration module for execution by the circuitry to register the storage device and the one or more properties with the system BIOS.

[0142] According to some examples for the example first device, the one or more properties for the storage device may be identified access rights for the storage device. For these examples, the identified access rights may include read-only access rights or read-write access rights.

[0143] In some examples, the example first device may be capable of supporting BIOS EDD services to provide at least one piece of information to the OS kernel or the one or more device drivers to access the RDMA STag stored in non-volatile memory. For these examples, the one or more parameters may be used to connect to the remote server or the one or more determined properties for the storage device.

[0144] According to some examples for the example first device, the system BIOS, the OS kernel, and the one or more device drivers may be configured to use a BIOS interrupt call 13 hex (INT 13h.) service.

[0145] In some examples of the example first device, the non-volatile memory at the network I / O device comprises at least one of 3-dimensional crosspoint memory, flash memory, ferroelectric memory, SONOS memory, polymer memory, nanowire, FeTRAM, FeRAM, nanowire, or EEPROM.

[0146] According to some examples of the example first device, the parameter module may receive the one or more parameters from non-volatile memory maintained at the network I / O device. For these examples, the one or more parameters include an IP address for the remote server, authentication information to authenticate the host device coupled to the network I / O device with the remote server, an identifier for the host device, or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0147] In some examples for the example first device, the parameter module may receive a first portion of the one or more parameters from non-volatile memory maintained at the network I / O device and a second portion of the one or more parameters from the remote server. For these examples, the first portion may include an identifier for a host device coupled to the network I / O device or authentication information to authenticate the host device to the remote server. The second portion may include an IP address for the remote server or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0148] According to some examples for the example first device, the RDMA protocol may include one of iWARP, Infiniband, or RoCE.

[0149] In some examples, example first methods implemented at a network I / O device may include receiving one or more parameters to enable the network I / O device to connect to a remote server over a network communication link. The example first methods may also include connecting to the remote server using the one or more parameters and establishing a control path to an NVMe controller managed at the remote server using an RDMA protocol. The example first methods may also include receiving an RDMA STag from the NVMe controller to access an OS kernel and one or more device drivers, and storing the RDMA STag in non-volatile memory at the network I / O device that is accessible to a system BIOS for a host device coupled to the network I / O device.Storing the RDMA STag may enable the system BIOS to use the RDMA STag to access the storage device and load the OS kernel and one or more device drivers to remotely boot the host device.

[0150] According to some examples of the example first methods, the RDMA STag may represent an assigned portion of the memory device that may be accessible using the RDMA STag to access the OS kernel and the one or more drivers.

[0151] In some examples of the example first methods, the allocated portion of the storage device that may be accessible using the RDMA STag may be jointly accessible by one or more other host devices connected to the remote server via respective separate network communication links.

[0152] According to some examples of the exemplary first methods, the allocated portion of the storage device accessible using the RDMA STag may include a first portion and a second portion. For these examples, the first portion may be read-only and shared by one or more other host devices connected to the remote server via respective separate network communication links. The second portion may be read / write and accessible only to the host device.

[0153] In some examples of the example first method, the second portion may include information associated with the host device and used to load the OS kernel or the one or more device drivers for execution by the circuitry at the host device.

[0154] According to some examples of the example first methods, the information assigned to the host device may include information to configure the host device as one or more of a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server.

[0155] In some examples, the example first methods may also include relinquishing control of the connection to the remote server via the network communication link to the OS kernel or the one or more device drivers. For these examples, the RDMA STag stored in non-volatile memory at the network I / O device may also be accessible to the OS kernel or the one or more device drivers to facilitate access to the storage device by the OS kernel or the one or more device drivers via use of the RDMA STag.

[0156] According to some examples, the example first methods may also include receiving the one or more parameters from non-volatile memory maintained at the network I / O device. For these examples, the one or more parameters may include an IP address for the remote server, authentication information to authenticate the host device to the remote server, an identifier for the host device, or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0157] In some examples, the example first methods may also include receiving a first portion of the one or more parameters from non-volatile memory maintained at the network I / O device and receiving a second portion of the one or more parameters from the remote server. For these examples, the first portion may include an identifier for the host device or authentication information to authenticate the host device to the remote server. The second portion may include an IP address for the remote server or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0158] According to some examples of the exemplary first methods, the RDMA protocol may include one of iWARP, Infiniband, or RoCE.

[0159] In some examples, the example first methods may also include determining one or more properties for the storage device controlled by the NVMe controller and registering the storage device and the one or more properties with the system BIOS.

[0160] According to some examples of the exemplary first methods, determining the one or more properties for the storage device may include identifying access rights to the storage device. For these examples, the identified access rights may include read-only access rights or read-write access rights.

[0161] In some examples, the example first methods may also include supporting the BIOS EDD service to provide at least one of information to the OS kernel or the one or more device drivers to access the RDMA STag stored in non-volatile memory, the one or more parameters used to connect to the remote server, or the one or more determined properties for the storage device.

[0162] According to some examples of the example first methods, the system BIOS loads the OS kernel and the one or more device drivers using a BIOS interrupt call 13 hex (INT 13h.) service.

[0163] In some examples, a first at least one machine-readable medium comprises a plurality of instructions that, in response to execution at a network I / O device coupled to a host device, cause the network I / O device to receive one or more parameters to enable the network I / O device to connect to a remote server over a network communication link. The instructions may also cause the network I / O device to connect to the remote server using the one or more parameters and to establish a control path to an NVMe controller managed at the remote server using an RDMA protocol. The instructions may also cause the network I / O device to determine one or more properties for a storage device controlled by the NVMe controller.The commands may also cause the network I / O device to register the storage device and the one or more properties with a system BIOS for the host device. The commands may also cause the network I / O device to receive an RDMA STag from the NVMe controller representing a mapped portion of the storage device that may be accessible using the RDMA STag. The mapped portion may store an OS kernel and one or more device drivers. The commands may also cause the network I / O device to store the RDMA STag in non-volatile memory at the network I / O device that is accessible to a system BIOS for the host device to enable the system BIOS to use the RDMA STag to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the host device.

[0164] According to some examples for the first at least one machine-readable medium, the instructions may also cause the network I / O device to relinquish control of the connection to the remote server via the network communication link to the OS kernel or the one or more device drivers. For these examples, the RDMA STag stored in non-volatile memory at the network I / O device may also be accessible to the OS kernel or the one or more device drivers to facilitate access to the storage device by the OS kernel or the one or more device drivers via use of the RDMA STag.

[0165] In some examples, the first at least one machine-readable medium may receive one or more parameters from non-volatile memory maintained at the network I / O device. For these examples, the one or more parameters may include an IP address for the remote server, authentication information to authenticate the host device to the remote server, an identifier for the host device, or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0166] According to some examples of the first at least one machine-readable medium, a first portion of the one or more parameters may be received from non-volatile memory maintained at the network I / O device, and a second portion of the one or more parameters may be received from the remote server. The first portion may include an identifier for the host device or authentication information to authenticate the host device to the remote server. The second portion includes an IP address for the remote server or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0167] In some examples of the first at least one machine-readable medium, the RDMA protocol includes one of iWARP, Infiniband, or RoCE.

[0168] According to some examples for the first at least one machine-readable medium, the determined one or more properties for the storage device may include identifying access rights to the storage device, wherein the identified access rights include read-only access rights or read-write access rights.

[0169] In some examples of the first at least one machine-readable medium, the system BIOS may access the OS kernel and the one or more device drivers using a BIOS interrupt call 13 hex (INT 13h) service.

[0170] According to some examples for the first at least one machine-readable medium, the commands also cause the network I / O device to support BIOS EDD services to provide the OS kernel or the one or more device drivers with at least one set of information to access the RDMA STag stored in non-volatile memory. For these examples, the one or more parameters can be used to connect to the remote server or the one or more determined properties for the storage device.

[0171] In some examples of the first at least one machine-readable medium, the allocated portion of the storage device accessible using the RDMA STag may be jointly accessible by one or more other host devices connected to the remote server via respective separate network communication links.

[0172] According to some examples of the at least one machine-readable medium, the allocated portion of the storage device accessible using the RDMA STag may include a first portion and a second portion. The first portion may be read-only and shared by one or more other host devices connected to the remote server via respective separate network communication links. The second portion may be read / write and accessible only to the host device.

[0173] In some examples of the first at least one machine-readable medium, the second portion may include information associated with the host device and used to load the OS kernel or the one or more device drivers for execution by circuitry at the host device.

[0174] According to some examples of the first at least one machine-readable medium, the information assigned to the host device may include information to configure the host device as one or more of a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server.

[0175] In some examples, an example second device for a network I / O device may include circuitry. The example second device may also include a parameter module, for execution by the circuitry, to receive one or more parameters associated with connecting to a remote client via a network communication link. The example second device may also include a connection module, for execution by the circuitry, to connect to the remote client via the network communication link using the one or more parameters. The example second device may also include a path module, for execution by the circuitry to establish a control path between the remote client and an NVMe controller maintained at a server coupled to the network I / O device, wherein the control path is established using an RDMA protocol.The example second device may also include a forwarding module for execution by the circuitry to forward one or more properties for a storage device controlled by the NVMe controller to the remote client, and to forward an RDMA Service Tag (STag) generated by the NVMe controller representing a dedicated portion of the storage device accessible using the RDMA STag. The dedicated portion may include an OS kernel and one or more device drivers. The example second device may also include a receive module for execution by the circuitry to receive the RDMA STag from the remote client and forward the RDMA STag to the NVMe controller to enable the remote client to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the remote client.

[0176] In some examples for the example second device, the one or more parameters may include an identifier for the remote client, authentication information to authenticate the remote client to the server, or network identification information for the network communication link to include a LAN identifier or a VLAN identifier.

[0177] According to some examples for the example second device, the parameter feature may receive a first portion of the one or more parameters from the remote client and receive a second portion of the one or more parameters from non-volatile memory maintained at the network I / O device. The first portion may include an identifier for the remote client or authentication information to authenticate the remote client to the server. The second portion may include an IP address for the server or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0178] In some examples for the example second device, the one or more properties for the storage device may include access rights granted to the remote client to access the storage device, where the granted access rights include read-only access rights or read-write access rights.

[0179] According to some examples for the example second device, the RDMA protocol may include one of iWARP, Infiniband, or RoCE.

[0180] In some examples of the example second device, the allocated portion of the storage device accessible using the RDMA STag may be jointly accessible by one or more other remote clients connected to the server via respective separate network communication links.

[0181] According to some examples for the exemplary second device, the allocated portion of the storage device accessible using the RDMA STag may include a first portion and a second portion. The first portion may be read-only and shared by one or more other remote clients connected to the server via corresponding network communication links. The second portion may be read / write and accessible only to the remote client.

[0182] In some examples of the example second device, the second portion may include information associated with the remote client and used to load the OS kernel or the one or more device drivers for execution by the circuitry at the remote client.

[0183] In some examples of the example second device, the information assigned to the remote client may include information to configure the remote client as one or more of a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server.

[0184] According to some examples of the example second device, the storage device may include a solid state drive (SSD) with non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, SONOS memory, polymer memory, nanowire, ferroelectric transistor random access memory (FeTRAM or FeRAM), nanowire, or electrically erasable and programmable read-only memory (EEPROM).

[0185] In some examples, the example second methods implemented at a network I / O device may include receiving one or more parameters associated with connecting to a remote client via a network communication link. The example second methods may also include connecting to the remote client via the network communication link using the one or more parameters and establishing a control path between the remote client and an NVMe controller managed at a server coupled to the network I / O device. The control path may be established using an RDMA protocol.The example second methods may also include forwarding one or more properties for a storage device controlled by the NVMe controller to the remote client and sending an RDMA STag generated by the NVMe controller representing an allocated portion of the storage device that may be accessible using the RDMA STag. The allocated portion may include an OS kernel and one or more device drivers. The example second methods may also include receiving the RDMA STag from the remote client and forwarding the RDMA STag to the NVMe controller to enable the remote client to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the remote client.

[0186] According to some examples of the example second methods, the one or more parameters may include an identifier for the remote client, authentication information to authenticate the remote client to the server, or network identification information for the network communication link to include a LAN identifier or a VLAN identifier.

[0187] In some examples, the example second methods may also include receiving a first portion of the one or more parameters from the remote client and receiving a second portion of the one or more parameters from a storage maintained at the server. The first portion may include an identifier for the remote client or authentication information to authenticate the remote client to the server, wherein the second portion includes an IP address for the server or network identification information for the network communication link, including a LAN identifier or a VLAN identifier.

[0188] According to some examples of the example second methods, the one or more properties for the storage device may include access rights granted to the remote client to access the storage device, wherein the granted access rights include read-only access rights or read-write access rights.

[0189] In some examples of the exemplary second methods, the RDMA protocol may include one of iWARP, Infinband, or RoCE.

[0190] In some examples of the example second methods, the allocated portion of the storage device accessible using the RDMA STag may be jointly accessible by one or more other remote clients connected to the server via respective separate network communication links.

[0191] According to some examples of the exemplary second methods, the allocated portion of the storage device accessible using the RDMA STag may include a first portion and a second portion. For these examples, the first portion may be read-only and shared by one or more other remote clients connected to the server via corresponding network communication links. The second portion may be read / write and accessible only to the remote client.

[0192] In some examples of the example second methods, the second portion may include information associated with the remote client and used to load the OS kernel or the one or more device drivers for execution by the circuitry at the remote client.

[0193] According to some examples of the example second methods, the information assigned to the remote client may include information to configure the remote client as one or more of a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server.

[0194] In some examples of the example second methods, the storage device may include an SSD with non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, SONOS memory, polymer memory, nanowire, FeTRAM, FeRAM, nanowire, or EEPROM.

[0195] In some examples, a second at least one machine-readable medium comprises a plurality of instructions that, in response to being executed at a network I / O device coupled to a server, cause the network I / O device to receive one or more parameters associated with connecting to a remote client over a network communication link. The instructions also cause the network I / O to connect to the remote client over the network communication link using the one or more parameters. The instructions also cause the network I / O to establish a control path between the remote client and an NVMe controller maintained at the server, wherein the control path is established using an RDMA protocol.The commands also cause the network I / O to forward one or more properties for a storage device controlled by the NVMe controller to the remote client and send an RDMA STag generated by the NVMe controller that represents a mapped portion of the storage device that may be accessible using the RDMA STag. The mapped portion may include an OS kernel and one or more device drivers. The commands also cause the network I / O to receive the RDMA STag from the remote client and forward the RDMA STag to the NVMe controller to enable the remote client to access the storage device and load the OS kernel and one or more device drivers to remotely boot the remote client.

[0196] According to some examples of the second at least one machine-readable medium, the RDMA protocol may include one of iWARP, Infiniband, or RoCE.

[0197] In some examples of the second at least one machine-readable medium, the allocated portion of the storage device accessible using the RDMA STag may be jointly accessible by one or more other remote clients connected to the server via respective separate network communication links.

[0198] According to some examples for the second at least one machine-readable medium, the allocated portion of the storage device accessible using the RDMA STag includes a first portion and a second portion. The first portion may be read-only and shared by one or more other remote clients connected to the server via corresponding network communication links. The second portion may be read / write and accessible only to the remote client.

[0199] In some examples of the second at least one machine-readable medium, the second portion may include information associated with the remote client and used to load the OS kernel or the one or more device drivers for execution by the circuitry at the remote client.

[0200] According to some examples of the second at least one machine-readable medium, the information assigned to the remote client may include information to configure the remote client as one or more of a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server.

[0201] It is emphasized that the Abstract of Disclosure is provided in accordance with 37 CFR Section 1.72(b), which requires an abstract that allows the reader to quickly determine the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Additionally, it is apparent from the foregoing Detailed Description that various features are grouped into a single example to streamline the disclosure. However, this method of disclosure should not be construed as reflecting an intent that the claimed examples require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in fewer than all of the features of a single disclosed example.The following claims are thus included in the detailed description, and each claim is considered a separate example. In the appended claims, the terms "including" and "wherein" are used as the unambiguous equivalents of the corresponding terms "comprising" and "wherein," respectively. Furthermore, the terms "first," "second," "third," etc., are used merely as identifiers and are not intended to impose numerical requirements on their objects.

[0202] Although the subject matter has been described in language specifically referring to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as exemplary forms for implementing the claims.

Claims

[1] Device (500) comprising: Circuits (520) for a network input / output, I / O, device (1200); a parameter module (522-1) for execution by the circuitry (520) to receive one or more parameters (505) to enable the network I / O device (1200) to connect to a remote server via a network communication link; a connection module (522-2) for execution by the circuitry (520) to connect to the remote server using the one or more parameters (505); a path module (522-3) for execution by the circuitry (520) to establish a control path to a non-volatile memory express, NVMe, controller managed at the remote server using a remote direct memory access, RDMA, protocol; and a receive module (522-6) for execution by the circuitry (520) to receive an RDMA service tag, STag, (515) from the NVMe controller and to access an operating system, OS, kernel and one or more device drivers. [2] The device (500) of claim 1, wherein the RDMA stage (515) represents a dedicated portion of a storage device controlled by the NVMe controller, which is accessible using the RDMA stage (515) to access the OS kernel and the one or more device drivers. [3] Device (500) according to claim 2, comprising: a storage module (522-7) for execution by the circuitry (520) to store the RDMA stage (515) in non-volatile memory in the network I / O device (1200) accessible through a system basic input / output system, BIOS, to a host device coupled to the network I / O device (1200), to enable the system BIOS to use the RDMA stage (515) to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the host device. [4] Device (500) according to claim 3, wherein: the connection module (522-2) relinquishes control of the connection to the remote server via the network communication link to the OS kernel or the one or more device drivers; and wherein the storage module (522-7) causes the RDMA stag (515) to be stored in non-volatile memory at the network I / O device (1200) to be accessible to the OS kernel or the one or more device drivers to facilitate access to the storage device by the OS kernel or the one or more device drivers via use of the RDMA stag (515). [5] Device (500) according to claim 3, comprising: a property module (522-4) for execution by the circuitry (520) to determine one or more properties for the storage device controlled by the NVMe controller; and a registration module (522-5) for execution by the circuitry (520) to register the storage device and the one or more properties with the system BIOS. [6] The apparatus (500) of claim 5, wherein the one or more properties for the storage device comprise identified access rights to the storage device, and wherein the identified access rights include read-only access rights or read-write access rights. [7] The apparatus (500) of claim 3, wherein the non-volatile memory in the network 1 / O device comprises at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon, SONOS, memory, polymer memory, nanowire, ferroelectric transistor random access memory, FeTRAM or FeRAM, nanowire, or electrically erasable and programmable read-only memory, EEPROM. [8] The device (500) of claim 1, comprising the RDMA protocol including one of Internet Wide Area RDMA Protocol, iWARP, Infiniband or RDMA over Converged Ethernet, RoCE. [9] Method (600) comprising: receiving (602) one or more parameters (505) at a network input / output, I / O, device (1200) to enable the network I / O device (1200) to connect to a remote server via a network communication link; connecting (604) to the remote server using the one or more parameters (505); establishing (606) a control path to a non-volatile memory express, NVMe, controller managed at the remote server using a remote direct memory access, RDMA, protocol; receiving (608) an RDMA service tag, STag, (515) from the NVMe controller to access an operating system, OS, kernel and one or more device drivers; and storing (610) the RDMA stage (515) in a non-volatile memory in the network I / O device accessible to a system basic input / output system, BIOS, for a host device coupled to the network I / O device (1200) to enable the system BIOS to use the RDMA stage (515) to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the host device. [10] The method (600) of claim 9, wherein the RDMA stage (515) represents a dedicated portion of the memory device accessible using the RDMA stage (515) to access the OS kernel and the one or more drivers. [11] The method (600) of claim 10, wherein the allocated portion of the storage device accessible using the RDMA stage (515) is commonly accessible by one or more other host devices connected to the remote server via respective separate network communication links. [12] The method (600) of claim 10, wherein the allocated portion of the storage device accessible using the RDMA stage (515) comprises a first portion and a second portion, and wherein the first portion is read-only and commonly accessible by one or more other host devices connected to the remote server via respective separate network communication links, and the second portion is read / write and accessible only to the host device. [13] The method (600) of claim 12, wherein the second portion includes information assigned to the host device and used to load the OS kernel or the one or more device drivers for execution by the circuitry in the host device. [14] The method (600) of claim 13, wherein the information assigned to the host device includes information to configure the host device as one or more of a database server, a file server, an email server, a print server, a web server, an application server, a game server, or a virtual server. [15] The method (600) of claim 9, comprising receiving the one or more parameters (505) from a non-volatile memory maintained at the network I / O device (1200), wherein the one or more parameters (505) include an Internet Protocol (IP) address for the remote server, authentication information to authenticate the host device to the remote server, an identifier for the host device, or network identification information for the network communication link to include a LAN identifier or a VLAN identifier. [16] The method (600) of claim 9, comprising receiving a first portion of the one or more parameters (505) from a non-volatile memory maintained at the network I / O device (1200) and receiving a second portion of the one or more parameters (505) from the remote server, wherein the first portion includes an identifier for the host device or authentication information to authenticate the host device to the remote server, and wherein the second portion includes an IP address for the remote server or network identification information for the network communication link to include a LAN identifier or a VLAN identifier. [17] Method (600) according to claim 9, comprising: determining one or more properties for the storage device controlled by the NVMe controller; and registering the storage device and one or more properties with the system BIOS. [18] The method (600) of claim 17, wherein determining the one or more properties for the storage device comprises identifying access rights to the storage device, and wherein the identified access rights include read-only access rights or read-write access rights. [19] The method (600) of claim 17, comprising: supporting BIOS EDD services to provide at least one of information to the OS kernel or the one or more device drivers to access the RDMA STag stored in the non-volatile memory, wherein the one or more parameters (505) are used to connect to the remote server or the one or more determined properties for the storage device. [20] Device (900) comprising: Circuits (920) for a network I / O device (1200); a parameter module (922-1) for execution by the circuitry (920) to receive one or more parameters (905) associated with connecting to a remote client via a network communication link; a connection module (922-2) for execution by the circuitry (920) to connect to the remote client via the network communication link using the one or more parameters (905); a path module (922-3) for execution by the circuitry (920) to establish a control path between the remote client and an NVMe controller managed at a server coupled to the network I / O device (1200), the control path being established using an RDMA protocol; a forwarding module (922-4) for execution by the circuitry (920) to forward one or more properties for a storage device controlled by the NVMe controller to the remote client and to forward an RDMA service tag, STag, (915) generated by the NVMe controller that represents an assigned portion of the storage device accessible using the RDMA Stag (915), the assigned portion including an OS kernel and one or more device drivers; and a receive module (922-5) for execution by the circuitry (920) to receive the RDMA stag (915) from the remote client and forward the RDMA stag (915) to the NVMe controller to enable the remote client to access the storage device and load the OS kernel and the one or more device drivers to remotely boot the remote client. [21] The apparatus (900) of claim 20, wherein the one or more parameters (905) comprise an identifier for the remote client, authentication information to authenticate the remote client to the server, or network identification information for the network communication link to include a LAN identifier or a VLAN identifier. [22] The apparatus (900) of claim 20, wherein the storage device includes a solid state drive, SSD, with non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon, SONOS, memory, polymer memory, nanowire, ferroelectric transistor random access memory, FeTRAM or FeRAM, nanowire, or electrically erasable and programmable read-only memory, EEPROM. [23] At least one machine-readable medium (1100) comprising a plurality of instructions that, in response to being executed at a network I / O device (1200) coupled to a server, cause the network I / O device (1200) to: receive one or more parameters (505) associated with connecting to a remote client via a network communication link; connect to the remote client via the network communication link using the one or more parameters (505); establish a control path between the remote client and a non-NVMe controller managed at the server, where the control path uses an RDMA protocol; forward one or more characteristics for a storage device controlled by the NVMe controller to the remote client and send an RDMA service tag, STag, (515) generated by the NVMe controller representing an assigned portion of the storage device accessible using the RDMA STag, the assigned portion including an OS kernel and one or more device drivers; and receive the RDMA stag (515) from the remote client and forward the RDMA stag (515) to the NVMe controller to enable the remote client to access the storage device and load the OS kernel and one or more device drivers to remotely boot the remote client. [24] At least one machine-readable medium (1100) according to claim 23, wherein the RDMA protocol includes one of Internet Wide Area RDMA Protocol, iWARP, Infiniband or RDMA over Converged Ethernet, RoCE. [25] At least one machine-readable medium (1100) according to claim 23, wherein the allocated portion of the storage device accessible using the RDMA stage (515) is jointly accessible by one or more other remote clients connected to the server via respective separate network communication links.

Images