Techniques for moving data between a network input / output device and a storage device

Abstract

Device comprising: Circuits (232) for a network I / O device (230); a receive module (422-2) for execution by the circuitry to receive a data frame (410) including a request to access a storage subsystem (236) comprised by the network I / O device, the storage subsystem including a solid state drive, SSD, (237); a buffer module (422-3) for execution by the circuits to use one or more buffers (215) maintained in a cache (214) for processor circuits (212) included in a processor socket (210) at a host device (205) coupled to the network I / O device, wherein the one or more buffers are used to exchange control information for the request, and wherein the control information is exchanged with a protocol stack (222) executed by the processor circuits; and a determination module (422-4) for execution by the circuitry to determine, based on the exchanged control information, whether data associated with the request should be read from or stored in the memory subsystem.

Description

TECHNICAL FIELD

[0001] Examples described here generally refer to storing or reading data from a storage subsystem based on a request enclosed in a receive data frame. BACKGROUND

[0002] Networking and storage are becoming increasingly intertwined as computing devices are deployed in highly distributed modes of operation. For example, stored data for a given network may be distributed across many computing devices, or network nodes. The given network may be configured as some type of cloud-based service, such as Software-As-A-Service (SAAS) or Infrastructure-As-A-Service (IAAS). Typically, storage and network subsystems are architecturally designed as separate subsystems for individual network nodes. Because they are located on separate subsystems, data arriving from the network at a given network node in storage, or vice versa, must be routed between the subsystems. Often, data can be routed through system storage to move the data between the two subsystems.

[0003] US 2009 / 0 292 861 A1 describes a network storage controller that uses a non-volatile solid-state storage (NVSSM) subsystem containing raw flash memory as stable storage for data, and uses remote direct memory access (RDMA) to access the NVSSM subsystem, including access to the flash memory. Data storage in the NVSSM subsystem is controlled by an external storage operating system in the storage controller. The storage operating system uses scatter-gather lists to specify RDMA read and write operations. Multiple client-initiated read or write operations can be combined in the storage controller into a single RDMA read or write operation, which can then be decomposed into multiple read or write operations and executed in the NVSSM subsystem.The memory accesses generated by a single RDMA read or write access can be directed to various storage devices in the NVSSM subsystem, which may include various forms of non-volatile solid-state memory. SUMMARY OF THE INVENTION

[0004] The object underlying the invention is achieved by the subject matter of the independent claims. Further advantageous embodiments are specified in the subclaims. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 illustrates a first exemplary system. Fig. Figure 2 illustrates a second exemplary system. Fig. 3 illustrates an example process. Fig. 4 illustrates an exemplary block diagram for a device. Fig. 5 illustrates an example of a first logic flow. Fig. Figure 6 illustrates an example of a second logic flow. Fig. Figure 7 illustrates an example of a storage medium. Fig. Figure 8 illustrates an example of a network input / output (I / O) device. DETAILED DESCRIPTION

[0005] As described in the present disclosure, a network node with a separate network and storage subsystem at which data is to be received or sent may route the data through system memory. This type of routing was originally used when network data capacities (e.g., bandwidth) were such that relatively small amounts of system memory were required to move the data between the subsystems. System memory was also required due to somewhat high access latencies for certain types of storage, including hard disk drives. However, as network data capacities have increased dramatically, larger amounts of system memory are required to maintain acceptable throughput. The overhead and latency associated with using system memory to route the data between the subsystems can further degrade performance.Furthermore, advances in non-volatile memory have enabled its use in storage subsystems. Non-volatile memory can be included in storage devices such as solid-state drives (SSDs), which have significantly lower access latencies. Therefore, larger network data capacities combined with lower access latencies for newer types of storage may result in the use of system memory to move data between network and storage subsystems as a less desirable and less efficient option for moving data. Because of these and other challenges, the examples described here are necessary.

[0006] In some examples, techniques for moving data between a network I / O device and a storage subsystem having one or more storage devices may be implemented. For these examples, circuitry for a network I / O device coupled to a host device may be capable of executing various modules to facilitate the movement of data. The various modules may include a receive module to receive a data frame including a request to access a storage subsystem managed at the network I / O device. The external storage may include a storage device such as a solid state drive (SSD). The various modules may also include a buffer module to utilize one or more buffers managed in a cache for processor circuitry included in a processor socket at the host device.The one or more buffers may be configured to exchange control information for the request. The control information may be exchanged using a protocol stack executed by the processor circuitry. The various modules may also include a determination module to determine whether data associated with the request should be read from or stored in the memory subsystem based on the exchanged control information.

[0007] In some other examples, the techniques may also include marking one or more buffers maintained in a cache for processor circuitry included in a processor socket at a host device. For these other examples, the one or more buffers may be marked by a network I / O device coupled to the host device to indicate use of the one or more buffers to provide data received or sent over one or more network connections coupled to the network I / O device. The data may be associated with requests to access a storage device controlled by a storage controller coupled to the host device. Additionally, for these other examples, a data frame may be received including a request by a remote device to access the storage device.Data associated with the request may then be forwarded to or received from the one or more marked buffers based on whether the request is to read the data from the storage device or to store the data in the storage device.

[0008] Fig. 1 illustrates a first exemplary system. As shown in Fig. 1, the first exemplary system includes a system 100 having a host device 105 coupled to a network I / O device 130 and a storage controller 140 via communication links 135 and 145. In addition, as shown in Fig. 1, the host device 105 may be capable of coupling to the network 150 through the NW I / O device 130 via the communication channel 140. In some examples, the remote device(s) 170 may establish one or more network connections through the communication channel 160 coupled to the network 150 and through the communication channel 140 coupled to the NW I / O device 130. For these examples, the one or more network connections between the remote device(s) 170 and the NW I / O device 130 may be used to receive data frames including requests to access a storage device (e.g., one of the storage devices 142) controlled by the storage controller 140 coupled to the host device 105.

[0009] According to some examples, the terms "host computer," "host device," "host," "network node," and "node" may be used interchangeably and may mean, for example, without limitation, one or more end stations, mobile internet devices, smartphones, media devices, I / O devices, tablet computers, appliances, intermediaries, network interfaces, clients, and / or portions thereof. Furthermore, in some examples, the terms "remote device" or "remote network node" may be used interchangeably and may mean, for example, without limitation, a computing device that can be remotely accessed (e.g., via a network connection) by a host device.

[0010] According to some examples, a "network" such as network 150 may include any mechanism, device, modality, and / or portions thereof that allow, facilitate, and / or enable two or more entities to be communicatively coupled to each other. In some examples, a first entity may also be "communicatively coupled" to a second entity if the first entity is capable of sending and / or receiving one or more commands and / or data to the second entity (e.g., enclosed in a data frame).

[0011] In some examples, as shown in Fig. 1, the host device 105 includes a processor socket 110. In addition, as shown in Fig. 1, the processor socket 110 includes the processor circuitry 112 and a last-level cache 114. The processor circuitry 112 may include one or more processing elements 112-1 through 112-n (where "n" represents any integer > 1) and a buffer manager 116. The processing elements 112-1 through 112-n may represent multiple processor cores or engines encapsulated in the processing circuitry 112, and the buffer manager 116 may be part of a device driver in an instantiation executed by the processing elements 112-1 through 112-n for an operating system for the host device 105. The processing elements 112-1 through 112-n may be capable of using the last level cache 114, which includes the buffers 115-1 through 115-m (where "m" represents any integer > 2), to at least temporarily provide or store data.According to some examples, buffers 115-1 through 115-m of last level cache 114 may either be individually assigned to a given processing element of processor circuitry 112, or they may be shared among the processor elements and may include memory types such as volatile memory with low access latency and relatively small data capacities (e.g., static random access memory (SRAM)). In either case, buffer manager 116 may represent logic and / or features executed by processing elements 112-1 through 112-n capable of managing access to buffers 115-1 through 115-m and facilitating the possible movement of data between NW I / O device 130 and the storage device(s) 142 controlled by memory controller 140. Facilitating the movement of data between NW I / O device 130 and the storage device(s) 142the storage devices 142 may include buffer manager 116 maintaining at least some mapping information to map commands associated with movement to one or more storage devices included in the storage device(s) 142.

[0012] According to some examples, NW I / O device 130 includes, as shown in Fig. 1, the circuitry 132 and the memory 134. For these examples, logic and / or features incorporated in various modules may be executed by the circuitry 132 to utilize the buffers 115-1 through 115-m of the last level cache 114 to facilitate the movement of data that is either read from the storage device(s) 142 or written / stored to the storage device(s) 142. The data movement may be responsive to data frames received from the remote device(s) 170, which may include requests to access the storage device(s) 142 via a read request or a write / store request. Additionally, the memory 134 at NW I / O device 130 may be configured to store firmware or software implemented by the various modules, or to at least temporarily maintain data associated with data frames received from the remote device(s) 170.received by the remote devices 170.

[0013] In some examples, logic and / or features at NW I / O device 130 may be capable of marking one or more of buffers 115-1 through 115-m to indicate the use of the one or more buffers for staging or at least temporarily storing data received or sent over one or more network connections routed between NW I / O device 130 and remote device(s) 170 through network 150. For these examples, buffers 115-1 through 115-m in last level cache 114 may be used to quickly move data between storage device(s) 142 and NW I / O device 130 with the assistance of buffer manager 116, rather than using system memory (not shown) at host device 105.Marking buffers 115-1 through 115-m may include the logic and / or features of NW I / O device 130 assigning an identifier to one or more of the buffers to indicate the use of the one or more marked buffers to access storage device 142. For example, buffer 115-1 may be marked, and a given identifier for that marked buffer may be communicated to buffer manager 116. Subsequently, when a request to access storage device(s) 142 is received in a data frame sent from remote device(s) 170 and received by NW I / O device 130, the given identifier may enable buffer manager 116 to quickly identify the specific buffer to use to provide data that may be moved based on the request.Additionally, the buffer manager 116 may be able to check the request to determine whether the request is for a read or write access and may provide the data accordingly.

[0014] According to some examples, a data frame may be received at NW I / O device 130 from a remote device among the remote devices 170. For these examples, the data frame may include a request by the remote device to access the storage device(s) 142. Logic and / or features at NW I / O device 130 may then forward or receive data associated with the request to or from the one or more tagged buffers based on whether the request is to read the data from the storage device(s) 142 or to store the data to the storage device(s) 142. For example, if the request is to read data from the storage device(s) 142, then the data may be provided in the tagged buffer (e.g., buffer 115-1) and then received from that tagged buffer by the logic and / or features of the NW I / O device 130 and then sent to the remote device that made the request.Alternatively, if the request is to store data in the storage device(s) 142, the data may be included in the data frame received from the remote device. This data may then be forwarded to the tagged buffer by the logic and / or features of the NW I / O device 130 to prepare the data for eventual storage in the storage device(s) 142. In some examples, the buffer manager 116 may reformat the request based on control information for the request so that the storage controller 140 can respond and fulfill the request accordingly.

[0015] In some examples, the communication links 135 and 145 that couple the NW I / O device 130 and storage controller 140 to host device 105 may be capable of operating in accordance with one or more industry standards associated with I / O interconnects, to include, but are not limited to, the Peripheral Component Interconnect (PCI) Express Base Specification, Revision 3.0, published in November 2010 (“PCI Express” or “PCIe”).

[0016] According to some examples, the storage device(s) 142 may include one or more devices in which data can be stored and / or retrieved. Additionally, for these examples, the storage device(s) 142 may include non-volatile memory for data storage. For example, the storage device(s) 142 may include, without limitation, one or more non-volatile electromechanical, magnetic, optical, and / or semiconductor storage devices. These devices may include hard disk drives (HDDs) or solid state drives (SSDs). The SSDs may include non-volatile types of memory such as 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon (SONOS) memory, polymer memory, nanowire, ferroelectric transistor random access memory (FeTRAM or FeRAM), nanowire, or electrically erasable programmable read-only memory (EEPROM).

[0017] According to some examples, the storage controller 140, the storage devices 142, and the communication link 145 may be capable of operating in accordance with the PCIe specification, as well as other industry standards associated with accessing non-volatile storage types such as SSDs using a PCIe-compliant communication link and / or protocols. These other industry standards may include, but are not limited to, the Non-Volatile Storage Express (NVMe) Specification, Revision 1.1, published in October 2012. The storage controller 140, the storage devices 142, and the communication link 145 may also be capable of operating in accordance with other industry standards, including, without limitation, the Serial ATA (SATA) Specification, Revision 3.1, published in July 2001, and the Serial Attached SCSI (SAS) Specification, Revision 2.1, published in December 2010, and / or Internet SCSI (iSCSI), Request for Comments 3720, published in April 2004.

[0018] In some examples, communication channel 140 may include one or more communication links over which network I / O device 130 may couple to network 150 and establish one or more network connections to the remote device(s) that couple to network 150 over communication channel 160, which may also include one or more communication links. These communication links included in communication channel 140 or 160 may include various types of wired, wireless, or optical communication media. For these examples, the communication links may operate according to one or more applicable communication or networking standards in any version. For example, the communication links may operate in accordance with one or more promulgated Institute of Electrical Engineers (IEEE) standards or specifications for wired or wireless networks.These standards are specifications that may include, without limitation: IEEE 802.11-2012 Standard for Information technology - Telecommunications and information exchange between systems - Local and metropolitan area networks - Specific requirements Part 11: WLAN Media Access Controller (MAC) and Physical Layer (PHY) specifications, published March 2012 and / or later versions of this standard ("IEEE 802.11") for wireless media or IEEE 802.3-2008, Carrier Sense Multiple Access with Collision Detection (CSMA / CD) Access Methods and Physical Layer Specifications, published December 2008 (hereafter "IEEE 802.3") for wired media.

[0019] In some examples, the NW I / O device 130 and the remote device(s) 170 may exchange data frames over the network 150 according to one or more protocols that may conform to and / or be compatible with various types of Remote Direct Memory Access (RDMA) protocols, such as Internet Wide Area RDMA Protocol (iWARP), Infiniband (IB) protocol, Ethernet protocol, TCP / IP protocol, and / or RDMA over Converged Ethernet (RoCE) protocol. For example, the iWARP protocol may conform to and / or be compatible with Recio et al., "An RDMA Protocol Specification," Internet Draft Specification, Internet Engineering Task Force (IETF), Oct. 21, 2002. Additionally, for example, the TCP / IP protocol may conform to and / or be compatible with the protocols described in Internet Engineering Task Force (IETF) RFC (RFC) 791 and 793, published September 1981. In addition, the IB protocol Infiniband™ Architecture Specification, Vol.2, Rev. 1.3, published in November 2012. In addition, for example, the RoCE protocol may conform to and / or be compatible with the Infiniband Architecture Specification Supplement, Vol. 1, Rev. 1.2.1, Appendix A16: "RDMA over Converged Ethernet (RoCE)," published in April 2010. The NW I / O device 130 and the remote device(s) 170 may also exchange data frames over the network 150 according to one or more protocols that may encapsulate Fibre Channel frames over Ethernet networks called Fiber Channel over Ethernet (FCOE). FCOE may be compatible with the protocols described in drafts of the American National Standard of Accredited Standards Committee INCITS T11 Technical Committee, Fibre Channel Backbone-5 (FC-BB-5) Standard, Revision 2.0, published in June 2009.Many different, additional and / or other protocols may be used for this data frame exchange without deviating from these examples (e.g., earlier and / or later developed versions of said related protocols and / or other protocols).

[0020] Fig. 2 illustrates a second exemplary system. As shown in Fig. 2, the second example includes a system 200. According to some examples, the system 200 includes a host device 205 coupled to a network I / O device 230 via a communication link 235. In some examples, the remote device(s) 270 may establish one or more network connections through communication channel 260 coupled to the network 250 and through communication channel 240 coupled to the network I / O device 230. For these examples, the one or more network connections between the remote device(s) 270 and the network I / O device 230 may be used to receive data frames including requests to access a storage subsystem (e.g., 236) managed and / or unified at the network I / O device 230.

[0021] According to some examples, the NW I / O device 230 may integrate both network and storage subsystems. As a result of integrating network and storage subsystems, requests to access the storage subsystem 236 may come either from the host device 205 via the communication link 235 or from the remote device(s) 270 via one or more network connections routed over the communication channel 240. For these examples, instead of providing data for requests made by the remote device(s) 270 to one or more buffers at a last-level cache (e.g., buffers 215-1 through 215-m of the last-level cache 214), the one or more buffers may be used to exchange control information for the request (e.g., data frame header information). The data may still be provided to the one or more buffers for requests originating from the host device 205 (e.g., from thethe applications 224). Regardless of the source of the requests, both movements of data do not involve the use of system memory (e.g., included in memory 220) to move data between the requesting agent and the storage subsystem to which the requesting agent requests access.

[0022] In some examples, as shown in Fig. 2, the host device 205 includes a processor socket 210 having similar components or features as for the host device 105 of Fig. 1. For example, processor socket 210 includes processor circuitry 212 including processing elements 212-1 through 212-n and a buffer manager 216. Processor socket 210 may also include a last-level cache 214 including buffers 215-1 through 215-n.

[0023] According to some examples, host device 205 may include memory 220 coupled to processor socket 210 via memory channel 215. For these examples, a protocol stack 222 and application(s) 224 may be maintained in memory 220 and may be capable of being executed by processor circuitry 212. Protocol stack 222 may include protocol processing software configured or arranged to process control information associated with data frames, including requests to access a storage subsystem, such as storage subsystem 236, which may include an SSD 237 located at NW I / O device 230. In some examples, the data frames may originate from logic and / or features of host device 205, such as application(s) 224, or they may originate from remote device(s) 270.The data frames may include control information through which protocol stack 222 may enable host device 205 to accept or reject a request to access storage subsystem 236.

[0024] According to some examples, logic and / or features incorporated in various modules may be executed by circuitry 232 at NW I / O device 230. One or more of these modules may use one or more buffers 215-1 through 215-m to exchange control information with protocol stack 222. For example, a data frame may be received at I / O device 230 from the remote device(s) 270, which may include control information in the form of a header for the data frame and may also include data in a payload. For this example, the control information included in the header may be sent to one or more buffers 215-1 through 215-m. Buffer manager 216 may then facilitate the exchange of this control information between protocol stack 222 and the one or more modules.A determination may then be made based on the exchanged information as to whether a request included in the data frame is intended to read data from or store data in the storage subsystem 236.

[0025] In some examples, a common or identical protocol format may be used for data frames received either from logic and / or features of host device 205 over communication link 235 or from the remote device(s) 270 over a network connection routed through communication channel 240. The same protocol format may include protocols such as FCOE, iWARP, Infiniband, or RoCE. For these examples, the various modules executed by circuitry 232 at NW I / O device 230 may receive requests to access storage subsystem 236 encapsulated using a same protocol format either from a local connection such as communication link 235 or over a network connection routed through communication channel 240 to network 250.

[0026] In some examples, the communication link 235 of system 200, similar to the communication links 135 and 145 of system 100, may be capable of operating in accordance with one or more industry standards or specifications associated with I / O interconnects to include PCIe. Furthermore, one or more of the modules executed by circuitry 232 at NW I / O controller 230 and at least SSD 237 of storage subsystem 236 may be capable of operating in accordance with the NVMe specification. Furthermore, the communication channels 240 or 260 may include various types of wired, wireless, or optical communication media. For these examples, the communication media, as well as the network 250, may operate in accordance with one or more applicable communication or networking standards, in any version, to include, but are not limited to, IEEE 802.11 or IEEE 802.3, to name a few.

[0027] Fig. 3 illustrates an example process 300. In some examples, elements of systems 100 or 200, as shown in the Fig. 1 or Fig. 2, may be used to illustrate exemplary operations associated with the flowchart for process 300 shown in Fig. 3. The described exemplary processes are not limited to implementations on systems 100 or 200 described in the Fig. 1 or Fig. 2 limited.

[0028] Transitioning from the start to decision block 310 (Data frame from host or remote device?), a data frame including a request to access storage subsystem 236 may be received from either host device 205 or remote device(s) 270. According to some examples, a request to access storage subsystem 236 may originate from application(s) 224, and therefore the data frame is considered to originate from host device 205. For these examples, the process transitions to block 320. Otherwise, if the data frame is received from remote device(s) 270, the process transitions to block 350.

[0029] Transitioning from decision block 310 to block 320 (Host Creates Control Information for Data Frame), protocol stack 222 may create control information for a data frame including a request for application(s) 224 to access storage subsystem 236. In some examples, elements of host device 205 may first verify / authenticate the request and then create the control information once the request has been validated, such as application(s) 224 having sufficient access privileges to access storage subsystem 236. For these examples, the control information may be associated with a protocol format commonly used for data frames including a request to access storage subsystem 236. For example, the common protocol format may be associated with FCOE, iWARP, Infiniband, or RoCE.The control information may, for example, be part of a header for the data frame to be routed to the NW I / O device 230.

[0030] Proceeding from block 320 to block 330 (exchange control information), there may be control information for the data frame from the application(s) 224 that may be exchanged between the protocol stack 222 and logic and / or features of the NW I / O device 230. According to some examples, one or more buffers 215-1 through 215-m may be used to exchange the control information. For example, the control information may include an indication of whether the application(s) 224 have requested read access or store access to the storage subsystem 236, and may also include further details regarding what data is to be read or how much data is to be stored. The NW I / O device 230 may also provide information to indicate whether the storage subsystem 236 has the capability to fulfill the access request.

[0031] Proceeding from block 330 to block 340 (host appends data as needed), protocol stack 222 may append or add data associated with the request to the data frame. In some examples, no data may be appended if the request is for a read access to storage subsystem 236. If the request is for a memory access, the data to be stored in storage subsystem 236 may be appended to the data frame.

[0032] Transitioning from decision block 310 to block 350 (NW I / O device splits control information and data), logic and / or features at NW I / O device 230 may split control information from the data frame received from the remote device(s) 270. According to some examples, the control information may be included in a portion of the header of the data frame. For these examples, the data frame may be in the protocol format commonly used for requests to access storage subsystem 236. For example, as noted above, the common protocol format may be associated with FCOE, iWARP, Infiniband, or RoCE.

[0033] Proceeding from block 350 to block 360 (exchanging control information), logic and / or features at NW I / O device 230 may exchange the split control information with protocol stack 222 at host device 205. In some examples, as noted above, one or more buffers 215-1 through 215-m may be used to exchange the control information. For example, the control information may include an indication of whether remote devices 270 have requested read or store access to storage subsystem 236, and may also include further details regarding what data is to be read or how much data is to be stored. NW I / O device 230 may also provide information to indicate whether storage subsystem 236 has the capability to fulfill the access request.

[0034] Proceeding from block 360 to block 370 (temporarily storing data associated with the request), logic and / or features at NW I / O device 230 may store data associated with the request. In some examples, if the data frames received from the remote device(s) 270 include data to be stored in the storage subsystem 236, the data may be stored at least temporarily in memory 236. For these examples, the data may be stored only temporarily to await a response to the request based on the information exchanged at block 360.

[0035] Starting at either block 340 or block 370 (host response for the request arrives), protocol stack 222 may send a response to the request to access storage subsystem 236 included in the data frame received at NW I / O device 230. According to some examples, the response may be based on the information exchanged at either block 330 or block 360.

[0036] Proceeding from block 375 to decision block 385 (Request Accepted?), logic and / or features at NW I / O device 230 may determine whether or not the request to access storage subsystem 236 has been accepted. In some examples, the request may be accepted based on the exchanged control information, indicating that storage subsystem 236 has the capability to service the request (e.g., has space available, has requested the data, or the requesting agent has sufficient access rights). If the request is accepted, the process proceeds to block 390. Otherwise, the method proceeds to block 395.

[0037] Transitioning from decision block 385 to block 390 (causing request to be satisfied), the request included in the receive data frame is caused to be satisfied and the process ends. According to some examples, logic and / or features at NW I / O device 230 may cause a controller (not shown) included with or at storage devices within storage subsystem 236 (e.g., SSD 237) to satisfy the access request. For examples where the request is from application(s) 224 to store data, logic and / or features at NW I / O device 230 may use one or more of buffers 215-1 through 215-m to receive or pull data temporarily provided to or stored by application(s) 224 in those buffers. For examples where the request is a store request from application(s) 224,the remote device(s) 270, the logic and / or features may pull the data from memory 237 that was used to temporarily store the data included in a data frame received from the remote device(s). For examples where the request is a read request from the application(s) 224 to read data from the storage subsystem 236, the read data may be pushed or provided to one or more buffers 215-1 through 215-m. For examples where the request is a read request from the remote device(s) 270, the read data may be enclosed in a data frame and sent to the remote device(s) 270 over the network connection routed via communication channels 240 and 260 and through the network 250.

[0038] Transitioning from decision block 385 to block 395 (Discard Data / Data Frame), if the request to access memory subsystem 236 is rejected, the data and / or data frames may be discarded, and the process ends. In some examples, any data temporarily stored in memory 236 from a data frame received from remote device(s) 270 may be discarded. In other examples, any data that may be provided by application(s) 224 to any of buffers 215-1 through 215-m may be discarded.

[0039] Fig. 4 illustrates an exemplary block diagram of a device 400. Although the device 400 shown in Fig. 4 has a limited number of elements in a particular topology, it is conceivable that the device 400 may include more or fewer elements in alternative topologies as desired for a given implementation.

[0040] The apparatus 400 may be supported by circuitry 420 managed at a NW I / O device coupled to a host device (e.g., circuitry 132 / 232 of NW I / O device 130 / 230). The circuitry 420 may be arranged to execute one or more software- or firmware-implemented components or modules 422-a. It should be noted that "a," "b," "c," and similar identifiers as used herein are intended to be variables representing any positive integer. For example, if an implementation specifies a value of a = 6, then a complete set of software or firmware for modules 422-a may include modules 422-1, 422-2, 422-3, 422-4, 422-5, or 422-6. The examples presented are not limited in this context and the different variables used throughout may represent the same or different integer values.

[0041] According to some examples, circuitry 420 may include a processor or processor circuitry. The processor or processor circuitry may be any of various commercially available processors, including, without limitation, AMD® Athlon®, Duron®, and Opteron® processors; ARM® application, embedded, and secure processors; IBM® and Motorola® DragonBall® and PowerPC® processors; IBM and Sony® Cell processors; Intel® Atom®, Celeron®, Core (2) Duo®, Core i3, Core i5, Core i7, Itanium®, Pentium®, Xeon®, Xeon Phi®, and XScale® processors; and similar processors. According to some examples, circuitry 420 may also be an application-specific integrated circuit (ASIC), and at least some modules 422-a may be implemented as hardware elements of the ASIC.

[0042] According to some examples, device 400 may include a tag module 422-1 for execution by circuitry 420. Tag module 422-1 may be capable of tagging one or more buffers maintained in a cache for processor circuitry included in a processor socket at a host device coupled to a network I / O device including device 400. Tagging the one or more buffers may include sending tag information 405, which may include one or more identifiers assigned to the one or more buffers by tag module 422-1. The identifiers may mark particular buffers as used for providing data received or sent over one or more network connections coupled to the network I / O device.The data may be associated with requests to access a storage device controlled by a storage controller coupled to the host device. For example, the storage device(s) 142 controlled by storage controller 140, as mentioned above for system 100 shown in FIG. Fig. 1. The tag module 422-1 may be capable of at least temporarily storing the parameter information 424-a (e.g., in a data structure such as a lookup table (LUT)). The buffer information 424-a may include the identifiers assigned to the tagged buffers.

[0043] In some examples, the device 400 may also include a receive module 422-2 for execution by the circuitry 420. The receive module 422-2 may be capable of receiving data frames 410 for a remote device, as shown for the system 100 in Fig. 1, or either by a remote device or an application executed by processor circuitry at the host device, as described for the system 200 as shown in Fig. 2. The receiving module 422-2 may be capable of at least temporarily storing the protocol information 425-b (e.g., in a LUT) to facilitate the reception of the data frame 410. The protocol information 425-b may enable the receiving module 422-2 to decode at least portions of the header of the data frame 410, which may be in the various protocol formats associated with access to a storage device or subsystem by a network I / O device. These various protocol formats may include, but are not limited to, the FCOE, iWARP, Infiniband, or RoCE protocol formats.

[0044] In some examples, apparatus 400 may also include a buffer module 422-3 for execution by circuitry 420. Buffer module 422-3 may be capable of using the one or more buffers maintained in the cache for the processor circuitry included in the processor socket for the host device. Buffer module 422-3 may be used to exchange control information included in control information 415 with a protocol stack executed by the processor circuitry at the host device. The exchanged control information may be for the access request included in data frame 410.For example, the control information 415 may include a header information split from the data frame 410 if a remote device sent the data frame 410, and may also include access information associated with the storage subsystem and / or storage device through which the access request was made. In other examples, if the data frame 410 originated from the host device (e.g., from an application), the control information 415 may also include access information associated with the storage subsystem and / or storage device through which the access request was made. In still other examples, the control information 415 may also include indications of whether the request is a read or a store request.For all of the above examples, the control information included in control information 415 may be passed between buffer module 422-3 and the protocol stack using the one or more buffers (e.g., based on tagged identifiers). Finally, based at least in part on the exchanged control information included in control information 415, request status 430 may be received to indicate whether the protocol stack and / or other elements of the host device accepted or rejected the request to access the storage subsystem and / or the storage device.

[0045] According to some examples, device 400 may also include a determination module 422-4 for execution by circuitry 420. Determination module 422-4 may be capable of determining whether data associated with the request should be read from or stored in the memory subsystem and / or the memory device. For these examples, determination module 422-4 may at least temporarily store control information 426-c, which includes control information in control information 415 exchanged between buffer module 422-3 and the protocol stack.

[0046] In some examples, apparatus 400 may also include a memory controller module 422-5 for execution by circuitry 420. Memory controller module 422-5 may be capable of controlling access to the memory subsystem and / or the memory device according to the determination made by determination module 422-4 and / or based on whether request status 430 indicates acceptance of the request.

[0047] According to some examples, the exchanged control information included in control information 415 may indicate a read, and request status 430 indicates acceptance of the request. For these examples, storage controller module 422-5 may cause the data to be read from the storage subsystem and / or storage device to be read as a data read operation 435. In some other examples, the exchanged control information may indicate storing, and request status 430 indicates acceptance of the request. For these other examples, storage controller module 422-5 may cause data included in data to be stored 440 to be stored to the storage subsystem and / or storage device.This data, included in the data to be stored 440, may either be retrieved from memory managed at the NW I / O device (temporarily stored by data included in data frame 410), or it may be pulled from one or more buffers through which the data was provided or temporarily stored when received from the requesting agents at the host device (e.g., applications - also stored by data included in data frame 410).

[0048] According to some examples, device 400 may also include a transmit module 422-6 for execution by circuitry 420. The transmit module 422-6 may be capable of transmitting data included in the data read operation 435 to the requesting agent that transmitted data frame 410. According to some examples, the transmit module 422-6 may be capable of using the protocol information 425-b to transmit the data included in the data read operation 435 in a data frame that may be in a protocol format that may include, but is not limited to, an FCOE, iWARP, Infiniband, or RoCE protocol format. The data frame may be sent to a requesting remote device agent over a network connection or sent to a requesting agent at the host device over a local connection that couples the NW I / O device to the host device.

[0049] Included herein is a set of logic flows representing exemplary methodologies for implementing new aspects of the disclosed architecture. While, for ease of explanation, the one or more methodologies shown herein are shown and described as a series of acts, one of ordinary skill in the art will understand that the methodologies are not limited by the order of the acts. Accordingly, some acts may occur in a different order and / or concurrently with other acts shown and described herein. One of ordinary skill in the art will understand that a methodology could alternatively be represented as a series of interrelated states or events, such as in a statechart diagram. Furthermore, not all acts illustrated in a methodology may be required for a new implementation.

[0050] A logic flow may be implemented in software, firmware, and / or hardware. In software and firmware embodiments, a logic flow may be implemented by computer-executable instructions stored on at least one non-transitory computer-readable medium or machine-readable medium, such as optical, magnetic, or semiconductor storage. The embodiments are not limited in this context.

[0051] Fig. Figure 5 illustrates an example of a first logic flow. As shown in Fig. 5, the first logic flow includes logic flow 500. Logic flow 500 may be some or all of the operations performed by one or more logic, features, or devices such as device 400 described herein. In particular, logic flow 500 may be implemented by one or more of tag module 422-1, receive module 422-2, buffer module 422-3, destination module 422-4, memory controller module 422-5, or transmit module 422-6.

[0052] According to some examples, at block 502, logic flow 500 may receive, at a network I / O device coupled to a host device, a data frame including a request to access a storage subsystem managed at the network I / O device, where the storage subsystem includes an SSD. For example, data frame 410 may be received by receive module 422-2 and may include a request to access a storage subsystem managed at the network I / O device, which may include device 400.

[0053] In some examples, at block 504, logic flow 500 may use one or more buffers maintained in a cache for processor circuitry included in a processor socket at the host device, wherein the one or more buffers are configured to exchange control information for the request, wherein the control information is exchanged with a protocol stack executed by the processor circuitry. For example, buffer module 422-3 may use the one or more buffers to exchange control information included in control information 415 with the protocol stack.

[0054] According to some examples, at block 506, logic flow 500 may then determine whether data associated with the request should be read from or stored in the storage subsystem based on the exchanged control information. For example, determination module 422-4 may use the control information included in control information 415 to determine the type of access. As mentioned above, additional modules of device 400 may then facilitate access to the storage subsystem and cause data to be stored in or read from the storage subsystem.

[0055] Fig. Figure 6 illustrates an example of a second logic flow. As shown in Fig. 6, the second logic flow includes logic flow 600. Logic flow 600 may represent some or all of the operations performed by one or more logic, features, or devices such as device 400 described herein. In particular, logic flow 600 may be implemented by one or more of tag module 522-1, receive module 522-2, buffer module 522-3, destination module 522-4, memory controller module 522-5, or transmit module 522-6.

[0056] According to some examples, at block 602, logic flow 600 may tag one or more buffers maintained in a cache for processor circuitry included in a processor socket at a host device. For these examples, the one or more buffers may be tagged by a network I / O device, including apparatus 500 that may be coupled to the host device, to indicate use of the one or more buffers for providing data received or sent over one or more network connections coupled to the network I / O device, where the data is associated with requests to access a storage device controlled by a storage controller coupled to the host device. For example, tag module 522-1 may provide tagging information 505 to tag the one or more buffers for use to provide receive data.

[0057] In some examples, at block 604, logic flow 600 may receive a data frame including a request from a remote device to access the storage device. For example, receive module 522-2 may receive data frame 510 including the request from the remote device.

[0058] According to some examples, at block 606, logic flow 600 may then forward or receive data associated with the request to or from the one or more marked buffers based on whether the request is to read the data from the storage device or to store the data in the storage device. For example, determination module 522-4 may determine whether the access request included in data frame 510 is for a read or a store to the storage device. Following a determination of a read, storage controller module 422-5 may cause data included in data read operation 435 to be read from the storage device, and this data may be provided to the one or more marked buffers. Buffer module 422-3 may then pull the data included in data read operation 435 from the one or more marked buffers and provide the data to send module 422-6.Module 422-6 may then send the data included in data read operation 435 to the remote device. Alternatively, upon determining to store, buffer module 422-3 may provide data included in data to be stored 440 to the one or more marked buffers. For this alternative, the provided data may have been obtained from data frame 410. Memory controller module 422-5 may then cause this provided data, included in data to be stored 440, to be stored in the memory device.

[0059] Fig. 7 illustrates an example of a computer-readable medium 700. The computer-readable medium 700 may comprise an article of manufacture. In some examples, the computer-readable medium 700 may include any non-transitory computer-readable medium or any machine-readable medium, such as optical, magnetic, or semiconductor memory. The computer-readable medium 700 may store various types of computer-executable instructions, such as instructions to implement logic flow 500 or logic flow 600. Examples of a computer-readable medium may include any physical medium capable of storing electronic data, for example, volatile memory or non-volatile memory, removable memory or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and so on.Examples of computer-executable instructions can include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, and the like. The examples are not limited in this context.

[0060] Fig. Figure 8 illustrates an example NW I / O device 800. In some examples, as shown in Fig. 8, the NW I / O device 800 may include a processing component 840, other platform components, or a communications interface 860. According to some examples, the NW I / O device 800 may be implemented in an NW I / O device coupled to a host or client device as mentioned above. In some examples, the NW I / O device 800 may or may not include a storage subsystem, further including a storage device such as an SSD.

[0061] According to some examples, processing component 840 may perform processing operations or logic for device 400 and / or computer-readable medium 700. Processing component 840 may include various hardware elements, software elements, or a combination of both. Examples of hardware elements may include devices, logic devices, components, processors, microprocessors, circuits, processor circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so on), integrated circuits, application-specific integrated circuits (ASICs), programmable logic units (PLDs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), memory units, logic gates, registers, a semiconductor device, chips, microchips, chipsets, and so on.Examples of software elements may include software components, programs, applications, computer programs, application programs, device drivers, system programs, software development programs, machine programs, operating system software, middleware, firmware, software modules, utilities, subroutines, functions, methods, procedures, software interfaces, application programming interfaces (APIs), instruction sets, computer code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.However, the determination of whether an example is implemented using hardware elements and / or software elements may vary in accordance with any number of factors, such as desired computational speed, power levels, thermal tolerances, processing cycle budget, input data speeds, output data speeds, memory resources, data bus speeds, and other design or performance constraints as desired for a particular implementation.

[0062] In some examples, other I / O components 850 may include, without limitation, various types of storage media in the form of one or more higher speed memory devices such as ROM, RAM, DRAM, DDRAM, SDRAM, SRAM, PROM, EPROM, EEPROM, flash memory, or any other type of storage medium suitable for storing information.

[0063] In some examples, the communication interface 860 may include logic and / or features to support a communication interface. For these examples, the communication interface 860 may include one or more communication interfaces operating according to various communication protocols or standards to communicate over direct or network communication links. Direct communications may occur via the use of communication protocols or standards described in one or more industry standards or specifications (including successors and variants) such as those associated with the PCIe specification, the NVMe specification, the RDMA protocol specification (e.g., iWARP, RoCE, Infiniband, the IEEE 802.3 or 802.11 specifications, RFC 791, RFC 793, or the FCoE standard).

[0064] The components and features of NW I / O device 800 may be implemented using any combination of discrete circuits, application-specific integrated circuits (ASICs), logic gates, and / or single-chip architectures. Furthermore, the features of NW I / O device 800 may be implemented using microcontrollers, programmable logic arrays, and / or microprocessors, or any combination of the foregoing, where appropriate. It should be noted that hardware, firmware, and / or software elements may be collectively or individually referred to herein as "logic" or "circuitry."

[0065] It is understood that the exemplary NW I / O device 800 shown in the block diagram of Fig. 8 may represent a functionally descriptive example of many potential implementations. Accordingly, any partitioning, omission, or inclusion of block functions depicted in the accompanying drawings does not imply that the hardware components, circuits, software, and / or elements for implementing those functions are necessarily partitioned, omitted, or included in the embodiments.

[0066] It is understood that the exemplary NW I / O device 800 shown in the block diagram of Fig.8 may represent a functionally descriptive example of many potential implementations. Accordingly, any partitioning, omission, or inclusion of block functions depicted in the accompanying drawings does not imply that the hardware components, circuits, software, and / or elements for implementing those functions are necessarily partitioned, omitted, or included in the embodiments.

[0067] One or more aspects of at least one embodiment may be implemented by representative instructions stored on at least one machine-readable medium that represent various logic within the processor that, when read by a machine, computing device, or system, cause the machine, computing device, or system to generate logic for performing the techniques described herein. Such representations, also known as "IP cores," may be stored on tangible machine-readable medium and delivered to various customers or manufacturing facilities, where they are loaded into manufacturing machines that manufacture the logic or processor.

[0068] Various examples can be implemented using hardware elements, software elements, or a combination of both. In some examples, hardware elements can include devices, logic devices, components, processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so on), integrated circuits, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), memory units, logic gates, registers, a semiconductor device, chips, microchips, chipsets, and so on.In some examples, software elements may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, utilities, subroutines, functions, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computer code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.However, the determination of whether an example is implemented using hardware elements and / or software elements may vary in accordance with any number of factors, such as desired computational speed, power levels, thermal tolerances, processing cycle budget, input data speeds, output data speeds, memory resources, data bus speeds, and other design or performance constraints as desired for a particular implementation.

[0069] Some examples may include an article of manufacture or at least one computer-readable medium. A computer-readable medium may include a non-volatile storage medium for storing logic. In some examples, the non-volatile storage medium may include one or more types of computer-readable storage media capable of storing electronic data, for example, volatile memory or non-volatile memory, removable memory or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and so on.In some examples, logic may include various software elements such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, utilities, subroutines, functions, methods, procedures, software interfaces, programming interfaces, instruction sets, computer code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.

[0070] According to some examples, a computer-readable medium may include a non-transitory storage medium for storing or maintaining instructions that, when executed by a machine, computing device, or system, cause the machine, computing device, or system to perform methods and / or operations according to the described examples. The instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. The instructions may be implemented according to a predefined computer language, manner, or syntax to instruct a machine, computing device, or system to perform a particular function. The instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0071] Some examples may be described using the phrase "in an example" or "an example," along with their derivatives. These terms mean that a particular feature, structure, or characteristic described in connection with the example is included in at least one example. The use of the phrase "in an example" in various places in the description does not necessarily always refer to the same example.

[0072] Some examples can be described using the terms "coupled" and "connected," along with their derivatives. However, these terms are not necessarily synonymous. For example, descriptions using the terms "connected" and / or "coupled" may indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" can also mean that two or more elements are not in direct contact with each other, but still work or interact with each other.

[0073] In some examples, an example apparatus for a network I / O device coupled to a host device may include circuitry. The example first apparatus may also include a receive module for execution by the circuitry to receive a data frame including a request to access a storage subsystem managed at the network I / O device. The external storage may include an SSD. The example first apparatus may also include a buffer module for execution by the circuitry to use one or more buffers managed in a cache for processor circuitry included in a processor socket at the host device. The one or more buffers may be used to exchange control information for the request, where the control information may be exchanged with a protocol stack executed by the processor circuitry.The exemplary first device may also include a determination module for execution by the circuitry to determine whether to read or store data associated with the request from or to the memory subsystem based on the exchanged control information.

[0074] According to some examples, the example device may also include a memory controller module to control access to the memory subsystem based on the determination by the determination module.

[0075] In some examples of the example device, the storage controller module and at least the SSD may be configured to operate in accordance with one or more industry standards to include the PCIe Base Specification, Revision 3.0 or the NVMe Specification, Revision 1.1.

[0076] According to some examples of the example apparatus, the receiving module may receive the data frame from the host device via a local connection to the host device or from a remote device via a network connection to the remote device, wherein the data frame is configured in accordance with a same protocol format regardless of whether it was received from the host device or remote device.

[0077] In some examples of the example device, the RDMA protocol may include one of FCoE, iWARP, Infiniband, or RoCE.

[0078] According to some examples of the example apparatus, the receiving module may receive the data frame from the host device, and the request included in the data frame is for an application executed by the processor circuitry to access the memory subsystem.

[0079] According to some examples of the example device, the request from the application may store data associated with the data frame in the storage subsystem. The data is at least temporarily stored in the one or more buffers prior to fulfilling the request to store the data in the storage subsystem.

[0080] In some examples of the example device, the request from the application may read data from the storage subsystem. The read data is stored in the one or more buffers, at least temporarily after the request to read the data from the storage subsystem is satisfied.

[0081] According to some examples of the example device, the control information for the request may include header information associated with the data frame to facilitate a determination by the protocol stack executed by the processor circuitry whether to accept or reject the request.

[0082] In some examples of the exemplary apparatus, the receiving module may receive the data frame from the remote device, and the request may be for the remote device to store data associated with the data frame in the storage subsystem. The data is stored at least temporarily in memory managed by the network I / O device and then caused to be stored in the storage subsystem following acceptance of the request.

[0083] According to some examples of the example apparatus, the receiving module may receive the data frame from the remote device and the request is for the remote device to read data from the storage subsystem.

[0084] In some examples, the example device may also include a transmit module for execution by the circuitry to transmit the read data to the remote device over the network connection following acceptance of the request.

[0085] According to some examples of the example apparatus, the local connection to the host device may be configured to operate in accordance with one or more industry standards, including PCIe Base Specification, Revision 3.0.

[0086] In some examples, example first methods implemented at a network I / O device coupled to a host device may include receiving a data frame including a request to access a storage subsystem managed at the network I / O device. For these examples, the external storage may include an SSD. The example first methods may also include using one or more buffers managed in a cache for processor circuitry included in a processor socket at the host device. The one or more buffers may be arranged to exchange control information for the request. The control information may be exchanged with a protocol stack executed by the processor circuitry.The example first methods may also include determining whether to read or store data associated with the request from or to the storage subsystem based on the exchanged control information.

[0087] According to some examples of the exemplary first methods, the data frame may be received from the host device via a local connection to the host device or from a remote device via a network connection to the remote device. For these examples, the data frame may be configured according to a similar protocol format regardless of whether it was received from the host device or the remote device.

[0088] In some examples of the exemplary first methods, the same protocol format may include FCoE, iWARP, Infiniband, or RoCE.

[0089] According to some examples of the example first methods, the data frame may be received by the host device and the request included in the data frame is for an application executed by the processor circuitry to access the memory subsystem.

[0090] In some examples of the exemplary first methods, the request from the application may be to store data associated with the data frame in the storage subsystem. The data is at least temporarily stored in the one or more buffers prior to fulfilling the request to store the data in the storage subsystem.

[0091] According to some examples of the exemplary first methods, the request from the application may be to read data from the storage subsystem. The read data is stored in the one or more buffers, at least temporarily after the request to read the data from the storage subsystem is satisfied.

[0092] In some examples of the example first methods, the control information for the request includes header information associated with the data frame that may be used to facilitate a determination by the protocol stack executed by the processor circuitry whether to accept or reject the request.

[0093] According to some examples of the exemplary first methods, the data frame may be received by the remote device, and the request is for the remote device to store data associated with the data frame in the storage subsystem. The data is stored at least temporarily in memory managed by the network I / O device and is then stored in the storage subsystem following acceptance of the request.

[0094] In some examples of the exemplary first methods, the data frame may be received by the remote device, and the request is for the remote device to read data from the storage subsystem. For these examples, the read data may be sent to the remote device over the network connection following acceptance of the request.

[0095] According to some examples of the example first methods, the SSD included in the storage subsystem may include non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon (SONOS) memory, polymer memory, nanowire, ferroelectric transistor random access memory (FeTRAM or FeRAM), nanowire, or electrically erasable programmable read-only memory (EEPROM).

[0096] In some examples of the example first methods, the local connection to the host device may be arranged to operate in accordance with one or more industry standards, including PCIe Base Specification, Revision 3.0.

[0097] In some examples, a first at least one machine-readable medium comprises a plurality of instructions that, in response to execution at a network I / O device coupled to a host device, cause the network I / O device to receive a data frame including a request to access a storage subsystem maintained at the network I / O device, wherein the storage subsystem includes an SSD. The instructions may also cause the network I / O device to use one or more buffers maintained in a cache for processor circuitry included in a processor socket at the host device. The one or more buffers may be arranged to exchange control information for the request. The control information may be exchanged with a protocol stack executed by the processor circuitry.The commands may also cause the network I / O device to determine whether data associated with the request should be read from or stored to the storage subsystem based on the control information exchanged.

[0098] According to some examples of the first at least one machine-readable medium, the data frame may be received from the host device via a local connection to the host device or from a remote device via a network connection to the remote device. The data frame may be configured according to a similar protocol format regardless of whether it was received from the host device or the remote device.

[0099] In some examples, the first at least one machine-readable medium may include the same protocol format FCOE, iWARP, Infiniband, or RoCE.

[0100] According to some examples, for the first at least one machine-readable medium, the data frame may be received by the host device, and the request included in the data frame is for an application executed by the processor circuitry to access the memory subsystem.

[0101] In some examples of the first at least one machine-readable medium, the request from the application may be to store data associated with the data frame in the storage subsystem. For these examples, the data is at least temporarily stored in the one or more buffers prior to satisfying the request to store the data in the storage subsystem.

[0102] According to some examples for the first at least one machine-readable medium, the request from the application may be to read data from the storage subsystem. The read data is stored in the one or more buffers, at least temporarily after the request to read the data from the storage subsystem is satisfied.

[0103] In some examples of the first at least one machine-readable medium, the control information for the request may include header information associated with the data frame to facilitate a determination by the protocol stack executed by the processor circuitry whether to accept or reject the request.

[0104] According to some examples, the first at least one machine-readable medium may receive the data frame from the remote device, and the request is for the remote device to store data associated with the data frame in the storage subsystem. The instructions may also cause the network I / O device to at least temporarily store the data in memory managed at the network I / O device and then cause the data to be stored in the storage subsystem following acceptance of the request.

[0105] In some examples, the first at least one machine-readable medium may receive the data frame from the remote device, and the request is for the remote device to read data from the storage subsystem. The commands may also cause the network I / O device to send the read data to the remote device over the network connection following acceptance of the request.

[0106] According to some examples of the first at least one machine-readable medium, the local connection to the host device may be configured to operate in accordance with one or more industry standards, including PCIe Base Specification, Revision 3.0.

[0107] In some examples, the example second methods implemented at a network I / O device coupled to a host device may include marking one or more buffers maintained in a cache for processor circuitry included in a processor socket at the host device. The one or more buffers may be marked by a network I / O device coupled to the host device to indicate use of the one or more buffers to provide data received or sent over one or more network connections coupled to the network I / O device. The data may be associated with requests to access a storage device controlled by a storage controller coupled to the host device.The example second methods may also include receiving a data frame including a request by a remote device to access the storage device. The example second methods may also include forwarding or receiving data associated with the request to or from the one or more marked buffers based on whether the request is to read the data from the storage device or to store the data in the storage device.

[0108] According to some examples of the example second methods, marking the one or more buffers may include associating an identifier with the one or more buffers to indicate using the one or more marked buffers to access the storage device based on one or more requests included in one or more data frames received from the remote device.

[0109] In some examples of the example second methods, the storage device includes an SSD, and the storage controller and the SSD may be configured to operate in accordance with one or more industry standards, to include the PCIe Base Specification, Revision 3.0 or the NVMe Specification, Revision 1.1.

[0110] According to some examples of the exemplary second methods, the SSD includes non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon (SONOS) memory, polymer memory, nanowire, ferroelectric transistor random access memory (FeTRAM or FeRAM), nanowire, or electrically erasable programmable read-only memory (EEPROM).

[0111] In some examples of the example second methods, the data frame may be configured in accordance with a protocol format to include FCOE, iWARP, Infiniband, or RoCE.

[0112] In some examples, a second at least one machine-readable medium comprises a plurality of instructions that, in response to being executed at a network I / O device coupled to a host device, cause the network I / O device to mark one or more buffers maintained in a cache for processor circuitry included in a processor socket at a host device. The one or more buffers may be marked by the network I / O device to indicate use of the one or more buffers to provide data received or sent over one or more network connections coupled to the network I / O device. The data may be associated with requests to access a storage device controlled by a storage controller coupled to the host device.The commands may also cause the network I / O device to receive a data frame that includes a request from a remote device to access the storage device. The commands may also cause the network I / O device to forward or receive data associated with the request to or from the one or more marked buffers based on whether the request is to read the data from the storage device or to store the data in the storage device.

[0113] According to some examples for the second at least one machine-readable medium, instructions to mark the one or more buffers may include instructions to cause the network I / O device to assign an identifier to the one or more buffers to indicate use of the one or more marked buffers to access the storage device based on one or more requests included in one or more data frames received from the remote device.

[0114] In some examples of the second at least one machine-readable medium, the storage device may include an SSD, and the storage controller and the SSD may be configured to operate in accordance with one or more industry standards to include the PCIe Basis Specification, Revision 3.0 or the NVMe Specification, Revision 1.1.

[0115] According to some examples of the second at least one machine-readable medium, the SSD may include non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon (SONOS) memory, polymer memory, nanowire, ferroelectric transistor random access memory (FeTRAM or FeRAM), nanowire, or electrically erasable programmable read-only memory (EEPROM).

[0116] In some examples of the second at least one machine-readable medium, the data frame is configured in accordance with a protocol format to include FCOE, iWARP, Infiniband, or RoCE.

Claims

[1] Device comprising: Circuits (232) for a network I / O device (230); a receive module (422-2) for execution by the circuitry to receive a data frame (410) including a request to access a storage subsystem (236) comprised by the network I / O device, the storage subsystem including a solid state drive, SSD, (237); a buffer module (422-3) for execution by the circuits to use one or more buffers (215) maintained in a cache (214) for processor circuits (212) included in a processor socket (210) at a host device (205) coupled to the network I / O device, wherein the one or more buffers are used to exchange control information for the request, and wherein the control information is exchanged with a protocol stack (222) executed by the processor circuits; and a determination module (422-4) for execution by the circuitry to determine, based on the exchanged control information, whether data associated with the request should be read from or stored in the memory subsystem. [2] Device according to claim 1, comprising: a memory controller module to control access to the memory subsystem based on the determination by the determination module. [3] The apparatus of claim 2, wherein the storage controller module and at least the SSD are configured to operate in accordance with one or more industry standards, including the PCIe Base Specification, Revision 3.0 or the NVMe Specification, Revision 1.

1. [4] The apparatus of claim 1, wherein the receiving module receives the data frame from the host device via a local connection to the host device or from a remote device via a network connection to the remote device, and wherein the data frame is configured in accordance with a same protocol format regardless of whether it was received from the host device or the remote device. [5] The apparatus of claim 4, comprising the same protocol format to include Fiber Channel over Ethernet, FCoE, Internet Wide Area RDMA Protocol, iWARP, Infiniband, or RDMA over Converged Ethernet, RoCE. [6] The apparatus of claim 4, wherein the control information for the request includes header information associated with the data frame to facilitate a determination by the protocol stack executed by the processor circuitry whether to accept or reject the request. [7] The apparatus of claim 6, wherein the receiving module receives the data frame from the remote device and the request is for the remote device to store data associated with the data frame in the storage subsystem, and wherein the data is at least temporarily stored in a memory managed at the network I / O device and then caused to be stored in the storage subsystem following acceptance of the request. [8] The apparatus of claim 7, wherein the receiving module receives the data frame from the remote device and the request is for the remote device to read data from the storage subsystem. [9] Device according to claim 8, comprising: a transmit module to be executed by the circuits to send the read data to the remote device via the network connection following acceptance of the request. [10] The apparatus of claim 8, wherein the local connection to the host device is configured to operate in accordance with one or more industry standards including the PCIe Base Specification, Revision 3.

0. [11] A method comprising: receiving (502) at a network I / O device a data frame including a request to access a storage subsystem comprised by the network I / O device, wherein the storage subsystem includes a solid state drive, SSD; using (504) one or more buffers maintained in a cache for processor circuits included in a processor socket at a host device coupled to the network I / O device, wherein the one or more buffers are configured to exchange control information for the request, and wherein the control information is exchanged with a protocol stack executed by the processor circuits; and determining (506), based on the exchanged control information, whether data associated with the request should be read from or stored in the storage subsystem. [12] The method of claim 11, comprising receiving the data frame from the host device via a local connection to the host device or from a remote device via a network connection to the remote device, wherein the data frame is configured in accordance with a same protocol format regardless of whether it was received from the host device or the remote device. [13] The method of claim 12, comprising the same protocol format to include Fiber Channel over Ethernet, FCoE, Internet Wide Area RDMA Protocol, iWARP, Infiniband, or RDMA over Converged Ethernet, RoCE. [14] The method of claim 12, wherein the data frame is received by the host device and the data frame included in the data frame is a request for an application executed by the processor circuitry to access the memory subsystem. [15] The method of claim 14, comprising requesting from the application to store data associated with the data frame in the storage subsystem, wherein the data is at least temporarily stored in one or more buffers prior to satisfying the request to store the data in the storage subsystem. [16] The method of claim 14, comprising requesting from the application to read data from the storage subsystem, wherein the read data is at least temporarily stored in the one or more buffers after satisfying the request to read the data from the storage subsystem. [17] The method of claim 11, wherein the SSD included in the storage subsystem comprises non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon, SONOS, memory, polymer memory, nanowire, ferroelectric transistor random access memory, FeTRAM or FeRAM, nanowire, or electrically erasable and programmable read-only memory, EEPROM. [18] At least one machine-readable medium (700) comprising a plurality of instructions that, in response to being executed at a network I / O device coupled to a host device, cause the network I / O device to: receive a data frame including a request to access a storage subsystem comprised by the network I / O device, wherein the storage subsystem includes a solid state drive, SSD; use one or more buffers maintained in a cache for processor circuitry included in a processor socket at the host device, wherein the one or more buffers are configured to exchange control information for the request, and wherein the control information is exchanged with a protocol stack executed by the processor circuitry; to determine, based on the exchanged control information, whether data associated with the request should be read from or stored in the storage subsystem. [19] At least one machine-readable medium according to claim 18, comprising receiving the data frame from the host device via a local connection to the host device or from a remote device via a network connection to the remote device, wherein the data frame is configured in accordance with a same protocol format regardless of whether it was received from the host device or the remote device. [20] At least one machine-readable medium according to claim 19, comprising the same protocol format to include Fiber Channel over Ethernet, FCoE, Internet Wide Area RDMA Protocol, iWARP, Infiniband or RDMA over Converged Ethernet, RoCE. [21] A method comprising: marking (602) one or more buffers maintained in a cache for processor circuits included in a processor socket at a host device, the one or more buffers being marked by a network I / O device coupled to the host device to indicate use of the one or more buffers for providing data received or sent over one or more network connections coupled to the network I / O device, the data being associated with requests to access a memory device comprised by a memory controller coupled to the host device; receiving (604) a data frame including a request by a remote device to access the storage device; and forwarding (606) or receiving data associated with the request to or from the one or more marked buffers based on whether the request is to read the data from the storage device or to store the data in the storage device. [22] The method of claim 21, wherein marking the one or more buffers includes associating an identifier with the one or more buffers to indicate using the one or more marked buffers to access the storage device based on one or more requests included in one or more data frames received from the remote device. [23] The method of claim 21, wherein the storage device comprises a solid state drive, SSD, and the storage controller and the SSD are configured to operate in accordance with one or more industry standards, including the PCIe Base Specification, Revision 3.0 or the NVMe Specification, Revision 1.

1. [24] The method of claim 23, wherein the SSD comprises non-volatile memory comprising at least one of 3-dimensional cross-point memory, flash memory, ferroelectric memory, silicon oxide nitride oxide silicon, SONOS, memory, polymer memory, nanowire, ferroelectric transistor random access memory, FeTRAM or FeRAM, nanowire, or electrically erasable and programmable read-only memory, EEPROM. [25] The method of claim 21, wherein the data frame is configured in accordance with a protocol format to include Fiber Channel over Ethernet, FCoE, Internet Wide Area RDMA Protocol, iWARP, Infiniband, or RDMA over Converged Ethernet, RoCE.

Images