Control plane fabric for communicating between a host and a shared network adapter

Abstract

Embodiments herein describe communication between a host and a shared network adapter. Control queues can be activated first in order to issue control commands to configure the data plane in the shared network adapter. If an error occurs, the system can suspend the data queues while allowing the control queues to still transfer control plane data between the host and the shared network adapter.

Description

Background Technology

[0001] This invention relates to a control plane architecture for establishing communication between a host and a shared network adapter.

[0002] A shared resource environment enables workloads running within that environment, or even workloads from different clients, to be consolidated onto a single machine, thereby allowing the machine's resources to be shared.

[0003] An example of shared resources is a shared network interface (e.g., a shared adapter), which facilitates communication with one or more hosts coupled to the shared network interface. A shared network interface facilitates the transfer of data, including large amounts of data, to or from the host and its file system. It also facilitates the streaming of other types of large data, such as video or complex engineering or scientific graphics. Transferring large amounts of data, whether files, streams, or other types, is very expensive in terms of central processing unit (CPU) cycles and network input / output (I / O) costs. Summary of the Invention

[0004] According to one embodiment of the present invention, a method includes activating a control queue between a host and a shared network adapter, issuing a control command to the shared network adapter using the control queue, wherein the control command configures a data plane for exchanging data between the host and the shared network adapter, activating the data queue between the host and the shared network adapter after issuing the control command, and suspending the data queue when an error is detected, while allowing the control queue to still transmit control plane data between the host and the shared network adapter.

[0005] A system according to an embodiment of the present invention includes: one or more processors configured to host a plurality of logical partitions or virtual machines; a shared network adapter configured to provide an interface between the plurality of logical partitions or virtual machines and a network interface card / controller (NIC); and main memory storing computer code configured to perform operations. The operations include activating a control queue between a host and the shared network adapter; issuing control commands to the shared network adapter using the control queue, wherein the control commands configure a data plane for exchanging data between the host and the shared network adapter; activating the data queue between the host and the shared network adapter after issuing the control commands; and suspending the data queue upon detecting an error, while allowing the control queue to continue transmitting control plane data between the host and the shared network adapter.

[0006] According to one embodiment of the present invention, a computer program product includes a computer-readable storage medium having computer-readable program code embodied therein, which is executable by one or more computer processors to perform operations. The operations include activating a control queue between a host and a shared network adapter; issuing control commands to the shared network adapter using the control queue, wherein the control commands configure a data plane for exchanging data between the host and the shared network adapter; activating the data queue between the host and the shared network adapter after issuing the control commands; and suspending the data queue upon detecting an error, while allowing the control queue to continue transmitting control plane data between the host and the shared network adapter. Attached Figure Description

[0007] Figure 1 A computing environment according to one embodiment is shown.

[0008] Figure 2 A shared resource environment according to one embodiment is shown.

[0009] Figure 3 A data structure of a data device according to one embodiment is shown.

[0010] Figure 4 This is a flowchart of a data structure for a data device according to one embodiment.

[0011] Figure 5 A queue description record according to one embodiment is shown.

[0012] Figure 6 An array of queue descriptors according to one embodiment is shown.

[0013] Figure 7A and 7B The TX and RX memory block page entries are shown according to one embodiment.

[0014] Figure 8 An index of the control program queue is shown according to one embodiment.

[0015] Figure 9 The RX completion queue item is shown according to one embodiment.

[0016] Figure 10 A shared resource environment according to one embodiment is shown.

[0017] Figure 11 This is a flowchart of an embodiment for activating a control queue and a data queue in a data device.

[0018] Figure 12It is a state machine for activating control queues and data queues in a data device, according to one embodiment.

[0019] Figure 13 This is a flowchart for configuring a data device according to one embodiment.

[0020] Figure 14 This illustrates a data device initialization process according to one embodiment.

[0021] Figure 15 This is a flowchart of an embodiment for activating a control queue and a data queue in a data device.

[0022] Figure 16 This is a flowchart of a method for transmitting commands using a control queue, according to one embodiment. Detailed Implementation

[0023] The embodiments described herein illustrate the implementation of communication between a host and a shared network adapter. Data communication can be established first through a control queue, which is then used to establish a data queue. For example, the control queue is part of the control plane, while the data queue is part of the data plane used to transfer data between the host and the shared network adapter. Advantageously, this makes it much easier than using two separate data devices to synchronize the data and control planes.

[0024] In one embodiment, when configuring the data device, the host first establishes data and control queues, but only activates the control queue. The control queue can then be used to transmit control commands from the host to the shared network adapter to configure the data plane. Once the data plane is configured, the host can activate the data queue, and the host and adapter can use the data queue to exchange data about TX and RX packets.

[0025] In one embodiment, the host and network adapter can deactivate the data plane while keeping the control plane enabled. For example, the network adapter might experience an error and deactivate the data queues, while the host and network adapter can still freely use the control queues to exchange control messages. Once the network adapter has recovered, it can restart the data plane. In this way, the network adapter and host can deactivate (or suspend) the data plane and data queues during operation without negatively impacting the control plane.

[0026] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or improvements to existing technologies in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0027] The following reference is made to embodiments presented in this disclosure. However, the scope of this disclosure is not limited to the specifically described embodiments. Rather, any combination of the following features and elements is contemplated for implementing and practicing the intended embodiments, regardless of whether different embodiments are involved. Furthermore, while the embodiments disclosed herein may achieve advantages over other possible solutions or prior art, whether a given embodiment achieves a particular advantage does not limit the scope of this disclosure. Therefore, the following aspects, features, embodiments, and advantages are merely illustrative and should not be considered elements or limitations of the appended claims unless expressly stated in the claims. Similarly, references to “the invention” should not be construed as a generalization of any inventive subject matter disclosed herein and should not be considered elements or limitations of the appended claims unless expressly stated in the claims.

[0028] Various aspects of this disclosure are described by narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). Regarding any flowchart, depending on the technology involved, operations may be performed in a different order than that shown in a given flowchart. For example, again according to the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0029] Computer Program Product Embodiment (“CPP Embodiment” or “CPP”) is a term used in this disclosure to describe any collection of one or more storage media (also referred to as “media”) collectively included in a collection of one or more storage devices, the collection of one or more storage devices collectively including machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A “storage device” is any tangible device capable of holding and storing instructions used by a computer processor. Without limitation, a computer-readable storage medium can be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punch cards or pits / platforms formed in the main surface of the disk), or any suitable combination of the foregoing. Computer-readable storage media, as used in this disclosure, should not be construed as storing transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media. As those skilled in the art will understand, data is typically moved at certain incidental points in time during the normal operation of the storage device, such as during access, defragmentation, or garbage collection; however, this does not make the storage device transient, because the data is not transient when it is stored.

[0030] The computing environment 100 includes examples of environments for executing at least some of the computer code involved in performing the methods of the present invention, such as establishing data constructs for data device 195, which allows operating system 122 to communicate with a shared adapter interface (discussed in more detail in the following figures). In addition to data device 195, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user equipment (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes processor assembly 110 (including processing circuitry 120 and cache 121), communication infrastructure 111, volatile memory 112, persistent storage device 113 (including operating system 122 and data device 195, as described above), peripheral device assembly 114 (including user interface (UI) device assembly 123, storage device 124, and Internet of Things (IoT) sensor assembly 125), and network module 115. Remote server 104 includes remote database 130. Public cloud 105 includes gateway 140, cloud coordination module 141, host physical machine set 142, virtual machine set 143, and container set 144.

[0031] Computer 101 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or to be developed in the future capable of running programs, accessing networks, or querying databases such as remote database 130. As is well known in the field of computer technology, and depending on the technology, the performance of a computer-implemented method can be distributed across multiple computers and / or multiple locations. On the other hand, in this presentation of computing environment 100, the detailed discussion focuses on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 can reside in the cloud, even... Figure 1 It is not shown in the cloud, and on the other hand, computer 101 does not need to be in the cloud unless it can be indicated with certainty to any extent.

[0032] Processor assembly 110 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed across multiple packages, such as multiple cooperating integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be readily accessible by the threads or cores running on processor assembly 110. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache in the processor assembly may be located “off-chip.” In some computing environments, processor assembly 110 may be designed to work with qubits and perform quantum computing.

[0033] Computer-readable program instructions are typically loaded onto computer 101 to cause the processor set 110 of computer 101 to perform a series of operational steps to implement a computer-implemented method, such that the instructions thus executed instantiate the method specified in the flowcharts and / or descriptive descriptions of the computer-implemented method included in this document (collectively, the "method of the invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the method of the invention. In computing environment 100, in block 400, at least some of the instructions for performing the method of the invention may be stored in permanent storage device 113.

[0034] Communication structure 111 is a signal transmission path that allows the various components of computer 101 to communicate with each other. Typically, this structure consists of switches and conductive paths, such as switches and conductive paths that form buses, bridges, physical input / output ports, etc. Other types of signal communication paths can be used, such as fiber optic communication paths and / or wireless communication paths.

[0035] Volatile memory 112 is any type of volatile memory now known or to be developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 112 is characterized by random access, but this is not necessary unless explicitly stated otherwise. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located externally relative to computer 101.

[0036] The persistent storage device 113 is any form of non-volatile memory for a computer, now known or to be developed in the future. The non-volatility of this memory means that the stored data is retained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage device 113. The persistent storage device 113 may be a read-only memory (ROM), but typically at least a portion of the persistent memory allows data to be written, deleted, and rewritten. Some common forms of persistent storage include hard disks and solid-state storage devices. The operating system 122 may take several forms, such as various known proprietary operating systems or operating systems employing an open-source portable operating system interface type with a kernel. The code included in the data device 195 typically includes at least some of the computer code involved in performing the methods of the present invention.

[0037] Peripheral device set 114 includes a set of peripheral devices for computer 101. Data communication connections between peripheral devices and other components of computer 101 can be implemented in various ways, such as Bluetooth connectivity, near field communication (NFC) connectivity, connections made by cables (such as Universal Serial Bus (USB) type cables), plug-in connections (e.g., secure digital (SD) cards), connections made through local area communication networks, and even connections made through wide area networks such as the Internet. In various embodiments, UI device set 123 may include components such as displays, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage device 124 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. Storage device 124 can be permanent and / or volatile. In some embodiments, storage device 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 requires substantial storage (e.g., where computer 101 locally stores and manages a large database), this storage can be provided by peripheral storage devices designed for storing very large amounts of data, such as a Storage Area Network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 comprises sensors that can be used in IoT applications. For example, one sensor could be a thermometer, while another could be a motion detector.

[0038] Network module 115 is a collection of computer software, hardware, and firmware that allows computer 101 to communicate with other computers via WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi transceiver, software for packetizing and / or depacketizing data transmitted over the communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the methods of the present invention can typically be downloaded to computer 101 from an external computer or external storage device via a network adapter card or network interface included in network module 115.

[0039] WAN 102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any technology known now or developed in the future for transmitting computer data. In some embodiments, WAN 102 may be replaced by and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include computer hardware such as copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0040] End User Equipment (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and can take any of the forms discussed above in conjunction with computer 101. EUD 103 typically receives useful and available data from the operation of computer 101. For example, assuming computer 101 is designed to provide recommendations to an end user, these recommendations are typically transmitted from network module 115 of computer 101 to EUD 103 via WAN 102. In this way, EUD 103 can display or otherwise present recommendations to the end user. In some embodiments, EUD 103 can be client equipment such as a thin client, heavy client, mainframe, desktop computer, etc.

[0041] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 can be controlled and used by the same entity operating computer 101. Remote server 104 represents a machine that collects and stores useful and available data used by other computers, such as computer 101. For example, if computer 101 is designed and programmed to provide recommendations based on historical data, that historical data can be provided to computer 101 from a remote database 130 of remote server 104.

[0042] Public cloud 105 is any computer system that can be used by multiple entities, providing on-demand availability of computer system resources and / or other computing capabilities (particularly data storage (cloud storage) and computing power) without the need for direct, active management by users. Cloud computing typically leverages resource sharing to achieve scalability consistency and economy. Direct and active management of the computing resources of public cloud 105 is performed by the computer hardware and / or software of cloud coordination module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments running on various computers constituting the host physical machine set 142, which is the entirety of physical computers in and / or available to the public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts as images or after the VCEs are instantiated. Cloud coordination module 141 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages the active instantiation of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that allow public cloud 105 to communicate via WAN 102.

[0043] Now, we will provide some further explanation of Virtualized Computing Environments (VCEs). A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from this image. Two common types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows multiple isolated user-space instances, called containers, to exist. From the perspective of the programs running within them, these isolated user-space instances typically appear as actual computers. Computer programs running on a regular operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running within a container can only use the contents of the container and the devices allocated to the container; this is a characteristic known as containerization.

[0044] Private cloud 106 is similar to public cloud 105, except that computing resources are available only to a single enterprise. While private cloud 106 is depicted as communicating with WAN 102, in other embodiments, private cloud may be completely disconnected from the Internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types) typically implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardization or proprietary technology that enables coordination, management, and / or data / application portability across the multiple component clouds. In this embodiment, public cloud 105 and private cloud 106 are both part of a larger hybrid cloud.

[0045] refer to Figure 2 An embodiment of a shared resource environment for incorporating and using one or more aspects of the present invention is described. In one example, the shared resource environment 200 is based on, for example, International Business Machines Corporation of Armonk, New York, and includes, for example, a System z® server also provided by International Business Machines Corporation. z / Architecture®, System®, and IBM® are registered trademarks of International Business Machines Corporation of Armonk, New York, USA. Other names used herein may be registered trademarks, common trademarks, or product names of International Business Machines Corporation (IBM) or other companies. Although z / Architecture® and System® are used as illustrative examples, the embodiments herein are not limited to this architecture and can be applied to, for example, Figure 1 Any suitable hardware system and operating system described herein. For example, the embodiments described herein can be applied to any computing system in which multiple entities (such as logical partitions (LPARs) or virtual machines (VMs) share the same adapter 210.

[0046] In this example, the shared resource environment 200 includes a central processing unit complex (CPC) 202, which has, for example, one or more partitions or regions 204 (e.g., logical partitions LPAR L1-LPAR L3, which may also be referred to as VMs). Each logical partition has a resident operating system 206, which may be different for the one or more logical partitions. That is, the operating system 206 can be of different types of operating systems. Although three logical partitions are described in this example, other embodiments may include more or fewer logical partitions. Furthermore, one or more partitions may not run an operating system, and / or may run an operating system other than that described herein. In addition, an LPAR or VM may have multiple operating systems.

[0047] One or more logical partitions are managed by hypervisor 250. Hypervisor 250 enables the hardware to virtualize LPARs.

[0048] Each LPAR is coupled to a shared network adapter 210. Adapter 210 includes a network interface card / controller (NIC) 212, which enables communication via an external network 214. The external network 214 is coupled to NIC 212 via port 216. Network 214 can be used to communicate between LPARs in the shared resource environment 200, or to communicate with processors in other processing environments via a local area network (LAN) or wide area network (WAN).

[0049] Adapter 210 may include software code (e.g., microcode) that communicates with the operating system in the LPAR. In other words, this code provides an interface for communication between the LPAR and the remaining components in adapter 210 (e.g., NIC 212). As discussed in more detail below, the LPAR also has data devices 195A-195C, which serve as an interface between the corresponding operating system in the LPAR and the shared network adapter 210. Although Figure 2 It is shown that each LPAR has one data device 195, but an LPAR (or the operating system in the LPAR) can have multiple data devices 195 (e.g., one data device 195 is dedicated to IPv4 services, while another data device 195 is dedicated to IPv6 services).

[0050] Adapter 210 also includes multiple data connections 218, each data connection coupled to one of the devices 195 within the LPAR. For example, data connection 218A is coupled to data device 195A in LPAR 1; data connections 218B and 218C are coupled to data devices 195B and 195C in LPAR 2, respectively; and data connection 218D is coupled to device 195D in LPAR 3. In one example, data connection 218 is an enhanced queued direct I / O (EQDIO) data connection. Furthermore, data connections 218 can be used to transmit data for user applications as well as control data.

[0051] Device 195A is further coupled to entity 222A in LPAR 1 (e.g., TCP / IP, for the OS protocol stack); data devices 195B and 195C are also coupled to entities 222B and 222C in LPAR 2 (e.g., client C1 and client C2); and device 195D is also coupled to virtual switch 224 in LPAR 3.

[0052] Virtual switch 224 enables further data sharing among entities 226A, 226B, and 226C of LPAR 3 (e.g., clients E1, E2, and E3). Virtual switch 224 includes multiple ports 228A, 228B, and 228C, each coupled to a corresponding client via NIC 230A, 230B, and 230C, respectively. The virtual switch allows clients coupled to it to communicate with each other without using adapters or external networks.

[0053] Figure 3 A data configuration of a data device according to one embodiment is illustrated. System 300 illustrates a data configuration in host memory 305, hardware system area (HSA) 310, and shared adapter 210. Host memory 305 may be memory in an LPAR or VM, while shared adapter 210 may include code (e.g., microcode) and memory serving as an interface between NIC 212 and LPAR (e.g., host).

[0054] The data construction in host memory 305 includes data device 195 (which is also established in shared adapter 210 during the process described in later figures), TX memory block page entry (SBPE) 320, control program (CP) queue index 325, RXSBPE 335, RX completion queue 340, and interrupt register 345. As described above, data device 195 establishes a connection between the host and shared adapter 210. Each LPAR (or each operating system in the LPAR) that wishes to use NIC 212 can create its own data device 195. Data device 195 may include any number of data queues, depending on... Figure 3 The data structure shown is designed to facilitate communication between the host and shared adapter 210. For example, a configurable number of data queues may exist, which can be any mixture of TX or RX queues. In one embodiment, at least one data plane TX queue and one data plane RX queue exist. Furthermore, data device 195 includes at least one pair of control queues (i.e., one control plane TX queue and one control plane RX queue).

[0055] The formats TX SBPE 320 and RX SBPE 335 will be... Figure 7A and 7BThis is discussed in more detail, but generally, these items contain pointers to packets sent from the host to the network via NIC 212 (in the case of TX SBPE 320), or pointers to packets received from the network at NIC 212 (in the case of RX SBPE 335). TX SBPE 320 may also contain pointers to control information used to pass configuration or debug-type information between the host and shared adapter 210. Each SBPE may indicate that packets are stored in one location, or packets may be stored in multiple different locations (e.g., the header may be stored in one location, while the payload is stored in another). Therefore, the pointers in SBPEs 320 and 335 can support either type of strategy for storing TX and RX packets in memory. Note that in one embodiment, the SBPE itself may store TX and RX packets, rather than having pointers to those packets.

[0056] The CP queue index 325 can indicate the number of TX SBPEs 320 ready to be processed by the shared adapter 210. For example, suppose a host has four packets it wants NIC 212 to send on network 214. The host can create four TX SBPEs 320 (e.g., TXSBPEs 0-3) for these four packets and then update the CP queue index 325 to notify the shared adapter 210 that the host has four packets ready to be sent. The shared adapter 210 can then read the CP queue index 325, identify the four TX SBPEs 320, and use the information in the four TX SBPEs 320 to retrieve the packets that will be sent by NIC 212. Figure 8 More details about CP queue index 325 are provided in the documentation.

[0057] In one embodiment, each TX queue and RX queue includes its own CP queue index 325 and adapter (ADP) ADP queue index 330.

[0058] The RX completion queue 340 maps the location of Ethernet packets in the RX data buffer. In one embodiment, the shared adapter 210 uses the completion queue 340 to indicate the arrival of a new RX packet in the NIC 212. Details of the RX completion queue 340 are as follows: Figure 9 Provided by China.

[0059] When an interrupt occurs, the shared adapter 210 sets the interrupt register 345. In one embodiment, register 345 includes an interrupt status (e.g., a one-byte value) that the host can set to indicate when an interrupt is needed. For example, if the host has finished processing all received packets, the CP in the host can set the interrupt status to indicate that it is idle. Therefore, if the shared adapter 210 receives additional packets, it can use the interrupt status to determine whether it should subsequently send an interrupt to the host. For example, if the interrupt status indicates that the host is not currently processing data in the RX queue, the shared adapter 210 can send an interrupt to the host. However, if the interrupt status indicates that the host is currently processing RX packets, no interrupt is needed.

[0060] Furthermore, in one embodiment, register 345 may include a bitmask, where each bit corresponds to a queue ID in a queue within the data device. Shared adapter 210 can use the bitmask to indicate which queues have data ready for processing by the host. While there may be multiple queues for data device 195, there may only be one interrupt register 345 for each data device 195.

[0061] HSA 310 is a dedicated storage area for hardware configuration tables. In this example, HSA 310 includes ADP queue index 330. In one embodiment, ADP queue index 330 is read / written by adapter 210 and is read only by CP. Shared adapter 210 updates ADP queue index 330. The TX ADP queue index is used by the adapter to indicate the completion of a control plane request or packet transmission. Although Figure 3 An ADP queue index 330 is shown, but in one embodiment, HSA 310 may store both the TX ADP queue index and the RX ADP queue index. During transmission, shared adapter 210 receives a TX completion notification and updates the TXADP queue index. Thus, TX ADP queue index 330 may store the next TXSBPE index to be consumed by shared adapter 210. Furthermore, updates to ADP queue index 330 by shared adapter 210 can be used to indicate the completion of a TX control or data request. CP queue index 325 indicates the next item to contain a new TX control or data request.

[0062] For the RX ADP queue index, shared adapter 210 updates the index to describe the received packets. The index can indicate the next RX data SBPE 335 that adapter 210 will consume and the next RX completion queue item 340 that adapter 210 will produce. The data SBPE index is updated when adapter 210 has used all allocated space in the data SBPE and ownership of the buffer space is returned to CP.

[0063] In addition to data device 195 and NIC 212, shared adapter 210 also includes adapter initiation index 350. When a host has a packet ready to send, it can use this data structure to interrupt or wake up shared adapter 210. Shared adapter 210 can use CP queue index 325 to retrieve TX SBPE 320 and program the NIC to retrieve the corresponding TX packet from host memory 305 using the pointer in TXSBPE 320. When using adapter initiation index 350, CP copies the contents of CP queue index 325 into adapter initiation index 350. Adapter 210 can then directly retrieve the SBPE corresponding to the queue index without retrieving CP queue index 325.

[0064] Figure 4 This is a flowchart of a method 400 for establishing a data structure for a data device according to one embodiment. For example, method 400 may configure a shared adapter to use... Figure 3 The various data constructs shown.

[0065] In block 405, the host establishes data structures for the data devices in host memory. These data structures may include, for example, TX SBPE 320, CP queue index 325, RX SBPE 335, RX completion queue entry 340, and interrupt register 345.

[0066] In block 410, the host transmits the Queue Description Record (QDR) to shared adapter 210. An example of a QDR is shown below. Figure 5 As illustrated, for example, the host may use different I / O protocols to communicate with the shared adapter before the control plane and data plane are established. An example I / O protocol could be a Channel Command Word (CCW), which defines the I / O operations used to communicate with the channel subsystem. The CCW contains channel commands, such as read, write, or control, and the data address of the data area involved. However, the embodiments described herein are not limited to any particular I / O protocol, as long as the protocol can transmit the QDR to the shared adapter.

[0067] In block 415, the shared adapter uses QDR to establish TX and RX queues for data services within the shared adapter. In one embodiment, the shared adapter first establishes TX and RX control queues for establishing the control plane, and then, after establishing the control plane, establishes TX and RX data queues for the data plane.

[0068] use Figure 5 The following is an example illustrating the information used to establish control and data queues. Figure 5The QDR 500 includes QFMT 505, version 510, queue descriptor size 515, TX queue count 520, RX queue count 525, system queue control area (SQCA) 530, interrupt reduction control (IRC) 535 (also referred to as "interrupt control"), queue information block (QIB) 540, and queue descriptor array item 545.

[0069] The Queue Format (QFMT) 505 enables the use of multiple protocols, each using a specific format for its SBPE, CP and ADP queue index areas, interrupt registers, etc.

[0070] Version 510 enables support for multiple versions of a specific QFMT. For example, version 2 can support extended formats of SBPE that were not supported in version 1.

[0071] The queue descriptor size 515 tells the shared adapter the size of the queue descriptor array item 545.

[0072] The TX queue count 520 tells the shared adapter 210 how many TX queues the data device has. In one embodiment, the QDR includes queue descriptor entries for each control queue and each data queue. For example, the RX control queue may be hardcoded as queue ID 0, and the TX control queue may be hardcoded as queue ID 1.

[0073] For data queues, a data device can have multiple TX queues, each with a different priority. For example, for each pass, a shared adapter may process at most 5 packets from one TX queue (even if it has more packets than that), but may process up to 10 packets from another higher-priority TX queue in the data device.

[0074] The RX queue count 525 informs the shared adapter 210 how many RX queues the data device has, which can include RX control queues and any number of RX data queues. Similar to TX queues, the data device can include multiple RX data queues, which can be assigned different priorities. The host can process received packets differently based on which RX queue they belong to. Another use of multiple RX queues is that the host can configure specific RX queues for specific service types (such as ARP), allowing the host to attach special programs designed specifically for that service type.

[0075] SQCA 530 includes for Figure 3The address of the CP queue index 325 in the host memory 305, i.e., the SQCA 530 contains the memory address of the CP queue index 325, allows the shared adapter 210 to read index 325 to determine which TX SBPE 320 in the host memory 305 corresponds to a new TX packet. In one embodiment, each TX queue in the data device has its own set of TX SBPE 320. Thus, each TX queue can also have its own CP queue index 325. The SQCA 530 may contain the address of each CP queue index of the data device. In one embodiment, the SQCA 530 is a 256-byte region, and each queue has a defined SQCA. For example, the first 8 bytes of the SQCA 530 are the queue index region associated with the defined queue. The SQCA address in the QDR may be a pointer to a 4K memory region containing 16 SQCAs. The QDR contains two of these pointers to represent a total of 32 possible queues (16 in each 4K page).

[0076] In some embodiments, if the predefined data size (e.g., 64 bits) is insufficient to store the address of each CP queue index (e.g., if the data drive has a bunch of TX queues), the QDR 500 may include multiple SQCA 530s.

[0077] IRC 535 Figure 3 The memory address of the interrupt register 345 is stored in the host memory 305, so that the shared adapter 210 can query the register 345 and interrupt the host when a newly received packet is ready for its processing, as described above.

[0078] The QIB 540 informs the shared adapter 210 of the attributes of the queues in the data device. The QIB 540 can contain information that is global to all queues. This can include things like host and adapter capabilities. Host capabilities include, for example, cache line size. Adapter capabilities could be protocol offloading, LAN speed, link aggregation, etc.

[0079] In this example, queue descriptor array item 545 can include different amounts of data (e.g., different sizes), which is why QDR 500 includes queue descriptor size 515. Details of queue descriptor array item 545 are provided in... Figure 6 As described in the text.

[0080] Figure 6 Show QDR (e.g., Figure 5The queue descriptor array entry 545 in QDR 500. In this example, queue descriptor array entry 545 includes queue ID 605, which identifies the specific queue in the data device corresponding to entry 545, and queue type 610, which indicates whether the corresponding queue is a control queue, RX queue, or TX queue, etc.

[0081] The queue descriptor array entry 545 also includes a storage block table (SBT) 615, a queue format record information block (QFRIB) 620, and an RX Ethernet packet completion queue (EPCQ) 625.

[0082] The SBT 615 stores the address of the location in host memory that corresponds to the TX SBPE or RX data SBPE of the queue. In this way, the shared adapter 210 knows where the TX SBPE 320 or RX data SBPE 335 is located in host memory.

[0083] The QFRIB 620 stores the address containing specific configuration information for either the TX or RX queue type.

[0084] The RX EPCQ 625 memory stores the address mapped to the location of the RX completion queue item 340. This allows the shared adapter 210 to know where the RX completion queue item 340 is located in host memory.

[0085] In this way, the host can Figure 5 The QDR 500 (which may include) Figure 6 The queue descriptor array entry 545 in the QDR 500 is sent to the shared adapter 210, so that the shared adapter 210 can use the data structure established by the host for the data device. That is, the shared adapter 210 can use the information in the QDR 500 to establish TX and RX queues that allow communication between the LPAR and the NIC.

[0086] Figure 7A A TX SBPE 320 according to one embodiment is shown. That is, Figure 7A This is an example of the TXSBPE 320 discussed above. As shown in the figure, the SBPE 320 includes R bits 705, I bits 710, type 715, flags 720, extended flags 725, error code 730, length 735, and CP buffer address 740.

[0087] Bit 710 identifies whether the packet represented by SBPE 320 is immediate data, where SBPE stores the TX packet instead of a pointer to the TX packet. This may only be valid for the data plane and not for the control plane.

[0088] Type 715 identifies the data type of the corresponding packet, such as 0x01 control packet or 0x02 Ethernet packet.

[0089] Flag 720 may be a linking flag to "link" multiple SBPEs together, where data packets are stored in different memory locations or to indicate that packets are stored in contiguous memory locations. Flag 720 may indicate that the shared adapter should generate an interrupt when a corresponding packet is processed. Flag 720 may also include an error flag, where error code 730 may contain a specific error code.

[0090] Extended flag 725 can be reserved for use by CPs running on the host.

[0091] Length 735 indicates the length of the control information or data pointed to by the buffer address in TX SBPE.

[0092] In one embodiment, CP buffer address 740 supports any byte-aligned address. However, buffer address 740 may be limited if buffer address 740 plus length 735 cannot cross a 4K boundary, and in such cases, a link should be used instead.

[0093] Figure 7B An RX SBPE according to one embodiment is shown. That is, Figure 7B Is it like this? Figure 3 An example of the RX SBPE 335 described in the figure. As shown, the SBPE 335 includes a CP buffer address 750, a CP buffer address extender 760, and a reserved space 770 for the CP flag.

[0094] The CP buffer address 750 and (optionally) the CP buffer address extender 760 can store a 64-bit CP buffer address. For example, bits 0 to 31 of the address can be stored in CP buffer address 750, while the remaining 32 to 63 bits of the address are stored in CP buffer address extender 760.

[0095] In one embodiment, the low-order bits (e.g., the 12 lowest-order bits) are reserved for the CP flag. Reserving the low-order bits forces the addresses in the SPBE to be 4K aligned. This is used to align the addresses with a memory page allocation algorithm that allocates memory in 4K blocks.

[0096] Figure 8 A CP queue index 325 according to one embodiment is shown. The CP queue index 325 may indicate the number of TX SBPEs 320 ready to be processed by the shared adapter 210. Index 325 includes an initial state 805, an SBPE index 810, and a completed SBPE index 815.

[0097] Initial state 805 tells the host whether the shared adapter is busy processing TX SBPE 320. That is, when the shared adapter is currently processing a TX SBPE, it can update initial state 805, so the host (e.g., LPAR) knows that the shared adapter is currently processing a TX SBPE. Therefore, if the host has more packets ready to transmit (e.g., creating an additional TX SBPE 32), the host can query initial state 805 to determine that the shared adapter is currently processing other TX packets and knows that it does not need to alert the adapter. When processing a TX packet is complete (i.e., when the shared adapter has completed all its outstanding work), the shared adapter can update initial state 805 to indicate that it is complete. Later, if the host has more TX packets to process, it can query initial state 805. If state 805 indicates that shared adapter 210 is idle, the host updates initial state 805 when it publishes new work and interrupts shared adapter 210, so adapter 210 knows that more TX packets are ready to be sent.

[0098] The SBPE index 810 can be updated to initiate data transfer from the host to the shared adapter. The completed SBPE index 815 is updated to indicate SBPEs that have been completed by the host and are ready for processing by the shared adapter. For example, if the host fills TXSBPE0-3, the host can update the completed SBPE index 815 to say "four," indicating that the host has filled SBPE0-3 and SBPE 4 is the next SBPE the host will use. This tells the shared adapter 210 that SBPE0-3 is ready to be processed. Once the CPSBPE index is updated, it transfers ownership of the TX SBPEs to the shared adapter. At this point, the TX SBPEs (one or more) become read-only to the CP.

[0099] In one embodiment, the host may also include an RX CP queue index. That is, although Figure 8 The TXCP queue index is shown, but the RX CP queue index can be used, which includes an initial state, a data SBPE index updated to allocate RX buffers, and an EPCQ SBPE index updated by the host to acknowledge RX packets. In one embodiment, the initial state in the CP TX and RX queue index areas works exactly the same. The initial state can tell when the CP index is updated and whether the shared network adapter needs to be initialized (e.g., whether the shared adapter should be interrupted). For the CP RX queue index, the shared network adapter can periodically check for new data SBPEs and EPCQs. In one example, the shared network adapter only sets the initial state to indicate that initialization is needed when one of these resources becomes unavailable. This prevents the shared network adapter from having to poll these areas.

[0100] Figure 9 This illustration shows an RX completion queue entry according to one embodiment. RX completion queue entry 340 maps the location of Ethernet packets in the RX data buffer. Entry 340 includes a T bit 905, a start alignment index 910, an RX data SBPE index 915, a flag 920, and a length 925.

[0101] Bit T 905 is a switching bit that will be switched by the shared network adapter each time the EPCQ wraps around. The CP can use this bit to detect new work without having to access the ADP RX queue index area.

[0102] The start alignment index 910 specifies the starting position of a packet within the RX buffer for an alignment unit (e.g., 256 bytes). In one embodiment, the alignment index alignment unit is based on the CP cache line size. This can be a programmable value controlled by the QIB.

[0103] The RX data SBPE index 915 contains the SBPE index associated with the RX storage block page entry 335 where the Ethernet packet is placed.

[0104] Flag 920 can include error and Ethernet packet type flags.

[0105] The length 925 can be specified according to the NIC RX descriptor to indicate the actual Ethernet frame length. This allows the CP to know the actual size of the Ethernet packets provided by the NIC.

[0106] In one embodiment, each EPCQ item represents the arrival of a specific Ethernet packet.

[0107] Figure 10 A shared resource environment according to one embodiment is illustrated. Environment 1000 includes many of the same components and elements discussed in the preceding figures, as indicated by the same reference numerals. Detailed descriptions of these components will not be repeated herein.

[0108] Environment 1000 illustrates a control plane 1005 and a data plane 1015 for data device 195. Control plane 1005 includes a control queue 1010 for the host and shared adapter 210 to exchange control data (e.g., control commands), while data plane 1015 includes a data queue 1020 for the host and shared adapter 210 to exchange data about TX and RX network packets.

[0109] In one embodiment, control queue 1010 and data queue 1020 are Figure 2An example of data connection 218 in the diagram could be, for example, a QDIO data connection. As discussed in more detail below, before configuring the QDIO connection, data device 195 may first use a different protocol (e.g., Channel Command Word (CCW) protocol) to facilitate communication between the host and shared adapter 210. Data device 195 can then be configured to use a different protocol (e.g., enhanced QDIO) that supports control queue 1010 and data queue 1020.

[0110] Advantageously, data device 195 may include control queue 1010 and data queue 1020, which differs from existing solutions where one data device (with a control plane queue) establishes the control plane and another data device (with a data plane queue) establishes the data plane. Synchronizing the two data devices is difficult. However, this is achieved in… Figure 10 This is avoided because data device 195 includes a control plane 1005 and a data plane 1015, which makes synchronization easier. (The following is incomplete and requires further context.) Figure 11 and 12 The interaction between the control plane 1005 and the data plane 1015 is discussed in more detail.

[0111] Furthermore, the "Activate Data Queue" control command enables the data queue architecture. This allows communication to be established between the OS and the shared network adapter used for data queues. In one embodiment, the system has the ability to activate a single data queue.

[0112] Environment 1000 also includes a CP 1025 in main memory 305, which includes microcode 1030. Microcode 1030 serves as an intermediate layer between the processor complex and the host's programmer-visible instruction set architecture (ISA). In one embodiment, microcode 1030 provides a common interface to shared adapter 210 for the operating system in the host. Thus, different operating systems (e.g., different types of operating systems) running in shared environment 1000 can communicate with shared adapter 210 using the same instructions provided by microcode 1030.

[0113] Figure 11 This is a flowchart of a method 1100 for activating a control queue and a data queue in a data device according to one embodiment. In block 1105, the host (or Figure 10 Microcode 1030 in the code activates the control queue between the host and the shared network adapter. In one embodiment, the host may have already established (or configured) both the control queue and the data queue for the data device, enabling the operating system in the host to communicate with the shared adapter; however, the host may only activate the control queue. That is, in block 1105, the data queue may be deactivated and data transfer may not be permitted.

[0114] The host can perform several operations to establish control queues and data queues for the data devices. The operations that can be performed before block 1105 to establish queues are as follows: Figure 13 The discussion is ongoing.

[0115] In block 1110, the host (or microcode) uses control queues to issue control commands to the shared network adapter. For example, in block 1110, the host may issue protocol control commands to configure the communication protocols to be used by the data plane. For example, the host may issue IP auxiliary primitives to control the type of information. IP auxiliary primitives can be used by the host to configure various data queues so that when an activation data queue is issued, the underlying hardware and / or software is appropriately configured to transmit packets from the appropriate TX data queue and receive packets in the appropriate RX data queue.

[0116] In one embodiment, the control plane is asynchronous, where the host (or CP) has the option to execute a single control command and wait for its response before issuing another command, or to issue multiple commands (SBPE) before processing any responses returned by the adapter. The method used will depend on the type of commands issued, whether the multiple commands are completely independent of each other, or the CP's preferences.

[0117] To track specific control operations, the CP can assign a unique sequence number to each command request issued on the output queue of the control plane. The adapter, in turn, returns that same sequence number in a response with the result of the request via the input queue of the control plane. This allows the CP to associate a response received on its control plane's input queue with a specific request previously issued via its control plane's output queue.

[0118] Some example control commands and IP auxiliary primitives that can be used to configure the data plane include: The CP stores subchannel data, where it retrieves any subchannel QDIO characteristics that may have been changed or modified by the hypervisor after successful control plane activation. In one embodiment, the CP determines whether the current environment requires a "subchannel ID" (SID) or a "subchannel token". The Exchange Adapter Control Command Request initiates communication to enable the control plane by informing the adapter of the CP of the management interface and its characteristics. The Set Group Parameter Request is used when grouping multiple adapters into logical groups. This command is typically used when binding multiple adapters to a single logical configuration. Initiate LAN request. If the NIC port is operable when the command is received, the adapter responds with an Initiate LAN response, at which point the CP can continue its data plane setup. The Adapter Parameters Request (ATR) retrieves supported hardware adapter commands that can be issued on this interface using the SETADAPTERPARMS IP auxiliary protocol command. These hardware commands give the CP the ability to change the operating parameters of the NIC card. A command to register one or more MAC addresses for connection to the interface. Commands that define a single virtual MAC address. This can include setting a base return code and a VLAN return code response for each VLAN ID set. Commands that define one or more VLAN IDs using a single MAC address. These commands may include setting primitive return codes and responses for VLAN return codes for each set of VLAN IDs. A command that defines multiple virtual Ethernet MAC addresses in a single primitive. In one embodiment, each virtual MAC address in the primitive has only a specified single VLAN. The response primitive may have a unique return code for each virtual MAC address and VLAN ID specified in the request. The command to delete a block of MAC addresses can, for example, remove multiple virtual Ethernet MAC addresses using a single primitive. The response can be used to set the raw return code as well as a specific return code associated with each specified virtual MAC address to be deleted. In the case of a management program, a command defines group MAC addresses, where a single group MAC can be defined as multiple VLAN IDs. If applicable, another command primitive can be used to delete a group MAC and its associated multicast IP address. Register to receive commands for services with a specified VLAN ID. The response primitive can be configured with a primitive return code and a return value. The command to set the group parameters (SETGROUPPARMS) for the logical group used for connection.

[0119] These are just a few examples of possible adapter control commands and IP auxiliary commands that can be transmitted on the control plane using control queues to configure the data plane of data devices.

[0120] In block 1115, the host (or CP and microcode) activates the data queue between the host and the shared network adapter. That is, once the protocol control commands establishing the data plane are complete, the CP requests the adapter to begin bidirectional data flow on the data plane. For example, the CP may send a command (using control queues) instructing the adapter to activate the defined data plane queues for inbound and outbound data transmission. In one embodiment, this command is the final instruction from the CP to the adapter, indicating that it is ready to accept data on the data plane and transmit data on the data plane. In the case of the Ethernet-IP protocol, this can include unicast, multicast, and broadcast packet transmissions.

[0121] In block 1120, the OS and shared adapter in the host use data queues to exchange data corresponding to TX and RX packets. This can include data in TX and RX data packets, data in SBPE, data in CP queue indexes, data in RX completion queue entries, data in ADP queue indexes, etc.

[0122] Figure 12 This is a state machine 1200 according to one embodiment for activating control queues and data queues in a data device. In state 1205, the host (or the CP in the host) activates the control plane. This is in Figure 11 It is described in detail in block 1105.

[0123] In state 1210, the host / CP resets the interface and executes only adapter management commands. That is, the data plane and data queues are not active. Whenever the host executes the aforementioned switch adapter control command request, this request initiates communication to enable the control plane by informing the adapter of the CP managing the interface and its characteristics, allowing the control plane to enter state 1210. In one embodiment, this command is a first control command issued by the CP to place the control plane in state 1210. In one embodiment, the shared adapter does not accept other control commands until it is in state 1210.

[0124] In one embodiment, the CP that executes the switch adapter control command request, while in any other state, forces the adapter to reset the data plane. This action returns the control plane to the state it was in when the control plane was initially activated. Inbound and outbound data transmission on the data plane is deactivated, and any adapter and protocol control settings are reset. The switch adapter control command is issued to reset all queues (control and data) and places the control plane in a state where the only executable control command is to initiate a LAN request / command.

[0125] In one embodiment, the control plane remains in state 1210 until the aforementioned LAN startup request is successfully completed. In one embodiment, while in state 1210, the OS can only issue adapter control commands. In one embodiment, while in state 1210, only the LAN startup primitive can be issued, which in turn activates the control queue to enable the acceptance of configuration control primitives through the control queue.

[0126] When the CP or adapter is ready to configure the communication protocol to be used by the data plane, it transitions the control plane to state 1220. When the Start LAN adapter control command completes successfully, the OS and adapter start the LAN. Once the OS transitions to the Start LAN state, it can begin issuing protocol control commands to configure the communication protocol to be used by the data plane. In the case of an Ethernet IP interface, the CP can issue IP auxiliary primitives for data transmission and any functions offloaded to the adapter. When the Start LAN control command completes, the shared network adapter signals to the OS that it is ready to receive protocol control commands to configure the communication protocol to be used on the data plane. It essentially opens the control queue for communication using protocol control commands and works similarly to the "Activate Data Queue" command used for the data queue. In one embodiment, only IDX (Switching Adapter) and Start LAN control commands are accepted in the control queue before Start LAN is complete.

[0127] The adapter can boot the LAN after recovering from an error that required resetting previously configured protocol settings by the CP. Typically, this happens when a NIC port becomes inoperable due to optical loss issues on the physical port. In this case, the adapter pre-changes the control plane to state 1215 when the port becomes inoperable and the adapter issues a stop LAN control command to the OS. The shared network adapter can reset back to the boot LAN state at any time by issuing a stop LAN primitive. This clears the configuration previously defined by the control primitive issued after the last successful boot LAN control primitive.

[0128] When the LAN startup command completes successfully, the CP can reissue the necessary protocol control commands to restart the data plane after an error. To resume transmission on the data plane, the CP can issue another data queue activation control command to enter a fully operational state 1225 for the data plane.

[0129] In addition to the data plane (and data queues) being operational in state 1225, when the control plane is in state 1225, the CP can issue any type of control command to modify the data plane environment in which it operates.

[0130] If, for any reason, the adapter or CP wants to suspend data transmission on the data plane queue without affecting the current protocol configuration, the adapter or CP issues a deactivate data queue adapter control command. For example, the adapter might experience an error it wants to recover from and need to suspend the data queue until it does so. Upon successful completion of the request, all data plane transmissions cease, and the control plane transitions to state 1230. The control plane remains in this state until it transitions back to being fully operational to the adapter or CP that issued the data queue activation request, or back to a state like 1210 or 1215 that requires further reconfiguration. While the control plane is in state 1230, in one embodiment, the CP may issue adapter control commands only to the adapter.

[0131] Furthermore, issuing a switch adapter control command request at any time allows the CP to reset itself and restart whenever an unexpected "protocol violation" return code is received from the adapter. This action allows the CP to reset the control plane to a known state so that it can start again from the data plane initialization. Issuing a switch adapter control command resets all queues (control and data) and puts the control plane in a state where the only executable control command is to initiate a LAN request / command.

[0132] Figure 13 This is a flowchart of a method 1300 for configuring a data device according to one embodiment. In one embodiment, method 1300 in Figure 11 Executed before block 1105, for example, method 1300 can be used to establish the data connection before activating the data connection (e.g., control and data queues) as described in method 1100.

[0133] In one embodiment, prior to starting method 1300, the OS in the host can clear or terminate any previous interface connections.

[0134] In block 1305, the OS in the host determines the type of data device. For example, the OS may determine (or sense) whether the data device is capable of supporting the data structures and data connections discussed above in the preceding figures. If the OS determines in block 1310 that the data device is incompatible (e.g., the data device is an older version), then method 1300 ends. For example, the previous data device does not support data structures and data connections such as... Figure 10 The control queue and data queue are shown.

[0135] However, if the data device is compatible with the data structures and techniques discussed in the previous figures, method 1300 proceeds to block 1315, where the OS determines the capabilities of the shared network adapter. For example, the OS may determine the maximum number of queues supported by the adapter and any other relevant characteristics.

[0136] In block 1320, the OS generates a QDR for the shared network adapter. For example, by determining the adapter's characteristics, the OS can generate a QDR for configuring as described above. Figure 5 The queues supported by the adapter described in [the document].

[0137] In block 1325, the OS establishes control and data queues in the shared network adapter. For example, as... Figure 4 As discussed, the adapter can use data in the QDR to establish queues and identify data structures in host memory (e.g., TX / RXSBPE, CP queue index, TX completion queue item, etc.). Furthermore, the OS can assign and configure device controls to enable interface connectivity for adapter interrupts.

[0138] Method 1300 can then proceed to block 1105 to begin. Figure 11 Method 1100.

[0139] Figure 14 A data device initialization process 1400 according to one embodiment is illustrated. Process 1400 is divided into three parts: CCW mode 1405, control plane active state 1420, and data plane active state 1425. CCW mode 1405 is used for communication between the control plane and the data plane with the data device and is inactive. CCW is merely one example I / O protocol that can be used. Any I / O protocol that defines I / O operations that can be used to communicate with the channel subsystem can be used.

[0140] In CCW mode 1405, in block 1410, the host uses QDR (e.g., the above in...). Figure 5 The QDR 500 (discussed in this document) is used to establish control plane queues. In one embodiment, the shared adapter first establishes TX and RX control queues to use the information in the QDR 500 to establish the control plane. In other words, the information in the QDR 500 is used to configure the TX and RX control queues in the data device, which the host and the shared network adapter can then use to exchange control commands and requests (e.g., IP auxiliary primitives).

[0141] At block 1415, the host activates the TX and RX control queues established at block 1410. This causes the control plane to become active, moving process 1400 to control plane active state 1420. There, the state machine (e.g., Figure 12 The state machine 1200 can be used to establish, configure, and then activate the data queues for the data plane. In other words, in order to activate the data plane of the data device, which allows the OS in the host and the shared network adapter to exchange data, state machine 1200 describes the use of the control plane in the data device.

[0142] In one embodiment, activating the data queue may include a LAN startup command. This could be a first IP auxiliary primitive sent by the OS. In response, the shared network adapter starts the LAN. Once started, other control commands can be exchanged. These commands could be protocol management configuration commands, such as those in... Figure 11 The protocol management configuration command is discussed at block 1110 of method 1100.

[0143] Once the data plane is active, process 1400 moves the data plane to active state 1425. That is, both the control plane and the data plane are active. However, in block 1430, the host or shared network adapter deactivates the data queue due to an error (e.g., suspends the data plane). Examples of errors caused by the OS or shared network adapter deactivating the data queue will be discussed later. Figure 15 This will be discussed in more detail later.

[0144] In one embodiment, deactivating the data queue deactivates the data plane, but its previous configuration remains intact. That is, deactivating the data queue does not necessarily reset its configuration. For example, the data queue can be deactivated to give the shared network adapter time to recover from an internal error. Once recovered, the shared adapter can restore the data queue without having to redistribute any adapter and protocol management commands used to configure the data plane (because its previous configuration is preserved). However, for other errors, the OS or the shared network adapter can reset the data plane's configuration, so the data plane should be reconfigured before restoring the data queue. This is discussed in the next section of process 1400.

[0145] In block 1435, the LAN can be stopped as part of the deactivation data queue. In one embodiment, stopping the LAN transitions to a pending processing state, which erases (or resets) the previous or current configuration of the data plane. In this case, the host and shared network adapter can reconfigure the data plane before resuming the data queue.

[0146] Figure 15 This is a flowchart according to one embodiment for activating control queues and data queues in a data device. In block 1505, the host (or...) Figure 10 In block 1025, the control queue between the host and the shared network adapter is activated. In one embodiment, the host may have already established (or configured) both the control queue and the data queue for the data device, enabling the operating system on the host to communicate with the shared adapter; however, the host may only activate the control queue. That is, in block 1505, the data queue may be deactivated and data transfer may not be permitted.

[0147] The host can perform several operations to establish control queues and data queues for the data devices. In the above... Figure 13 The section discusses example operations that can be performed before block 1505 to establish a queue.

[0148] In Block 1510, the host or CP uses control queues to issue control commands to the shared network adapter. For example, in Block 1510, the host may issue protocol control commands to configure the communication protocols to be used by the data plane. For example, the host may issue IP auxiliary primitives to control the type of information. IP auxiliary primitives can be used by the host to configure various data queues so that when an activation data queue command is issued, the underlying hardware and / or software is appropriately configured to send packets from the appropriate TX data queue and receive packets in the appropriate RX data queue. Examples of these IP auxiliary primitives (and adapter control commands) are provided in... Figure 11 The method described in block 1110 of 1100 may also include control commands. Figure 12 The adapter and protocol management commands discussed in the document.

[0149] In one embodiment, the control plane is asynchronous, where the host (or CP) has the option to execute a single control command and wait for its response before issuing another command, or to issue multiple commands (SBPE) before processing any responses returned by the adapter. The method used will depend on the type of command issued, whether the multiple commands are completely independent of each other, or the CP's preferences.

[0150] To track specific control operations, the CP can assign a unique sequence number to each command request issued on the output queue of the control plane. The adapter, in turn, returns that same sequence number in a response with the result of the request via the input queue of the control plane. This allows the CP to associate a response received on its control plane's input queue with a specific request previously issued via its control plane's output queue.

[0151] In block 1515, the host or CP activates the data queue between the host and the shared network adapter. That is, once the adapter and protocol management commands for establishing the data plane are complete, the CP requests the adapter to begin bidirectional data flow on the data plane. For example, the CP may send a command (using control queues) instructing the adapter to activate the defined data plane queues for inbound and outbound data transmission. In one embodiment, this command is the final instruction from the CP to the shared network adapter to be ready to receive and transmit data on the data plane. In the case of the Ethernet-IP protocol, this can include unicast, multicast, and broadcast packet transmissions.

[0152] In block 1520, the OS, CP, or shared network adapter identifies an error. Errors can include a NIC port becoming inoperable due to optical loss issues on the physical port, adapter errors, protocol violation codes, etc. In one embodiment, the error could be any error that should suspend the data plane.

[0153] In block 1525, the CP or OS suspends the data queue while allowing the control queue to continue transmitting control plane data. In one embodiment, the CP suspends data transmission on the data plane queue without affecting the current protocol configuration by issuing a deactivate data queue adapter control command. For example, all data plane transmissions stop, and the control plane can be switched to... Figure 12 The state shown is 1230. The control plane remains in this state until it transitions back to full operability, where the adapter or CP issues an activation data queue request, or returns to a state requiring further reconfiguration, such as... Figure 12 The states shown are 1210 or 1215.

[0154] In addition to pausing the data queues, the CP can reset the control plane to a known state, allowing it to restart data plane initialization. For example, issuing a switch adapter control command resets all queues (control and data) and places the control plane in a state where the only executable control command is to initiate a LAN request / command. In one embodiment, the data plane is reconfigured before it can be restored.

[0155] Figure 16 This is a flowchart of a method 1600 for transmitting commands using a control queue according to one embodiment. In block 1605, the OS in the host retrieves a local copy of the next available SBPE to be used to control an outbound operation. In one embodiment, the operation is a control command that the host wants to send to the shared adapter.

[0156] In block 1610, the CP or shared adapter returns the current CP queue index register from the SQCA of the control queue. SQCA is above... Figure 5 This was discussed in detail.

[0157] In block 1615, the OS generates control requests. For example, the OS may generate adapter or protocol management commands in the SBPE (Local CP Queue Index) on the output queue of the control plane. Examples of these control commands have been discussed in block 1110 of method 1100.

[0158] In block 1620, the OS increments the local CP queue index. This sets the local index of the next available SBPE for outbound transfers. In this example, "outbound transfer" on the control queue is from the perspective of the O / S sending adapter or protocol management commands to the shared network adapter.

[0159] In block 1625, the OS uses the Set CP Queue Index Control (SCPQC) instruction to store the updated local CP queue index. The SCPQC instruction is used to manage the ring's CP queue index so that data can be sent out on its output queue. In one embodiment, storing the updated local CP queue index along with the SCPQC instruction provides control commands from SBPE 0 to the shared adapter for transmission.

[0160] In block 1630, the CP or shared adapter sets a new CP queue index register in the queue's SQCA. Optionally, the CP also grants the adapter control over the processing queue.

[0161] In block 1635, the shared adapter generates a response to the control request generated by the OS. Furthermore, the shared adapter places the response on the next available SBPE in the control plane's input queue. The shared adapter can also indicate that a job decision (i.e., interrupt control) for that input queue is awaiting processing in the IRC. The CP uses the IRC to determine the input queue with pending jobs. If the IRC is in a "no-interrupt state," this method stops here. Otherwise, the interface is placed in a no-interrupt state in the IRC.

[0162] While the foregoing relates to embodiments of the present invention, other and further embodiments of the present invention may be designed without departing from the basic scope of the present invention, and the scope of the present invention is defined by the appended claims.

Claims

1. A method comprising: Activate the control queue between the host and the shared network adapter; The control queue is used to issue control commands to the shared network adapter, wherein the control commands configure a data plane for exchanging data between the host and the shared network adapter; After the control command is issued, the data queue between the host and the shared network adapter is activated; as well as When an error is detected, the data queue is paused, while the control queue is allowed to continue transmitting control plane data between the host and the shared network adapter.

2. The method according to claim 1, wherein, The control queue is part of the control plane in the data device, which acts as an interface between the operating system in the host and the shared network adapter.

3. The method according to claim 1, further comprising: After identifying the error: Receives a command to stop the LAN and returns to the LAN startup state, which cleared previously issued control commands.

4. The method according to claim 1, wherein, After identifying the error, the data queue is paused without affecting the current protocol configuration.

5. The method of claim 1, further comprising: After identifying the error: Reset the control plane; as well as Reconfigure the data plane.

6. The method according to claim 5, wherein, Resetting the control plane places it in a state where the only control command that can be executed is to initiate a LAN request.

7. The method according to claim 1, wherein, The error is a problem with the shared network adapter.

8. The method according to claim 1, wherein, The error is that the network interface card / controller (NIC) port changes from being operable to being inoperable.

9. A system comprising: One or more processors are configured to host multiple logical partitions or virtual machines; A shared network adapter is configured to provide an interface between the multiple logical partitions or virtual machines and the network interface card / controller NIC. as well as Host memory stores computer code configured to perform operations, said operations including: Activate the control queue between the host and the shared network adapter; The control queue is used to issue control commands to the shared network adapter, wherein the control commands configure a data plane for exchanging data between the host and the shared network adapter; After issuing the control command, the data queue between the host and the shared network adapter is activated; and When an error is detected, the data queue is paused, while the control queue is allowed to continue transmitting control plane data between the host and the shared network adapter.

10. The system according to claim 9, wherein, The control queue is part of the control plane in the data device, which acts as an interface between the operating system in the host and the shared network adapter.

11. The system according to claim 9, wherein, The operation further includes: after identifying the error: Receives a stop LAN command and returns to the start LAN state, which cleared previously issued control commands.

12. The system according to claim 9, wherein, After identifying the error, the data queue is paused without affecting the current protocol configuration.

13. The system according to claim 9, wherein, The operation further includes: after identifying the error: Reset the control plane; and Reconfigure the data plane.

14. The system according to claim 13, wherein, Resetting the control plane places it in a state where the only control command that can be executed is to initiate a LAN request.

15. A computer program product, the computer program product comprising: A computer-readable storage medium having computer-readable program code embodied therein, the computer-readable program code being executable by one or more computer processors to perform operations including: Activate the control queue between the host and the shared network adapter; The control queue is used to issue control commands to the shared network adapter, wherein the control commands configure a data plane for exchanging data between the host and the shared network adapter; After issuing the control command, the data queue between the host and the shared network adapter is activated; and When an error is detected, the data queue is paused, while the control queue is allowed to continue transmitting control plane data between the host and the shared network adapter.

16. The computer program product according to claim 15, wherein, The control queue is part of the control plane in the data device, which acts as an interface between the operating system in the host and the shared network adapter.

17. The computer program product according to claim 15, wherein, The operation further includes: after identifying the error: Receives a stop LAN command and returns to the start LAN state, which cleared previously issued control commands.

18. The computer program product according to claim 15, wherein, After identifying the error, the data queue is paused without affecting the current protocol configuration.

19. The computer program product according to claim 15, wherein, The operation further includes: after identifying the error: Reset the control plane; and Reconfigure the data plane.

20. The computer program product according to claim 19, wherein, Resetting the control plane places it in a state where the only control command that can be executed is to initiate a LAN request.

Images