Implementing advanced error handling for a shared adapter in a virtualized system

Abstract

A method for implementing extended fault collection for an input / output adapter (I / O adapter) in a computer system, wherein the I / O adapter is partitioned into a plurality of partitionable endpoints, each partitionable endpoint (PE) corresponding to a function, and wherein the method comprises an adapter PE associated with the I / O adapter and a plurality of virtual function PEs, and wherein the method comprises: in response to an adapter failure, blocking the adapter PE; Blocking the VF PEs associated with the adapter in response to a blocking of the adapter PE; Informing an adapter driver and VF device drivers about the error and initiating a recovery by the adapter driver and VF device drivers, and Blocking access to the VF device drivers, allowing the adapter recovery to complete successfully.

Description

Field of the invention

[0001] The present invention relates generally to the field of data processing and, more particularly, to a method, system, and computer program product for implementing enhanced error handling (EEH) for a hardware I / O adapter, such as a Single Root Input / Output Virtualization (SRIOV) adapter, in a virtualized system. Description of related technology

[0002] Single-root input / output virtualization (SRIOV) is a PCI standard that represents an adapter technology building block for I / O virtualization in the PCI Express (PCIe) industry. SRIOV capability is a feature of many new PCIe adapters for Fibre Channel, Ethernet, Infiniband, and converged network adapters (CNAs).

[0003] The SRIOV adapter features an I / O adapter virtualization architecture that allows a single I / O adapter to be shared across numerous different logical partitions simultaneously. Sharing occurs at a physical level, so each logical partition has access to a portion of the physical adapter. Sharing is achieved by partitioning the adapter into numerous different PCI functions and then distributing access to those functions.The adapter is presented as one or more physical functions (PFs) that control functions used for both configuration and I / O, for example, and as a set of virtual functions (VFs) used for I / O and, to a limited extent, configuration. Each VF represents a portion of the adapter's capacity that can be assigned to a logical partition independently of other VFs. Each logical partition has a device driver for each VF assigned to the logical partition.

[0004] Documents have already been published in this context. For example, document US 2013 / 0 275 972 A1 describes a method for controlling a virtual machine. Common I / O adapters are used for different virtual machines and coordinated via a hypervisor. Document EP 1 130 501 A1 describes a debug port for use by multiple data processing cores. This is a hardware solution that uses an emulation controller connected to an integrated circuit with multiple data processing cores. This allows multiple signal paths of the integrated circuit to be monitored. Furthermore, document US 2012 / 0 159 245 A1 describes enhanced error handling for self-virtualizing I / O units for logically partitioned data processing systems. Particular emphasis is placed on coordinating state recovery after errors between the logical partitions.Finally, document US 2012 / 0 254 862 A1 describes an efficient migration of virtual functions to ensure high availability and resource balancing. A virtual function driver, guest operating system, or virtual machine monitor includes a self-emulation layer that creates a shadow state that emulates invisible states of the virtual machine being migrated.

[0005] However, with a shared hardware I / O adapter, such as the SRIOV adapter, troubleshooting the shared adapter currently requires coordination across multiple partitions. Previous solutions only required coordination within a single partition, so a new solution is needed.

[0006] Therefore, there is a need for an effective mechanism to enable extended error handling (EEH) for a shared hardware I / O adapter, or SRIOV adapter, in a virtualized system. It is desirable that such a mechanism enable effective and efficient error handling operations to cover multiple partitions. SUMMARY OF THE INVENTION

[0007] This problem is solved by the subject matter of the independent patent claims. Further embodiments arise from the respective dependent patent claims.

[0008] Main aspects of the present invention are intended to provide a method, system, and computer program product for implementing extended error handling (EEH) for a hardware I / O adapter, such as an SRIOV adapter, in a virtualized system. Other important aspects of the present invention are intended to provide such a method, system, and computer program product with substantially no adverse effects and in a manner that overcomes many of the disadvantages of prior art arrangements.

[0009] In summary, a method, system, and computer program product are provided for implementing improved error handling for a hardware I / O adapter, such as an SRIOV adapter, in a virtualized system. The hardware I / O adapter is partitioned into multiple endpoints, with each partitionable endpoint (PE) corresponding to a function, and an adapter PE associated with the entire adapter. The endpoints are managed both independently for actions scoped to a single function and as a group for actions scoped to the entire adapter. An error or failure of the adapter PE blocks the adapter PE and is propagated to the VF PEs associated with the adapter, thereby blocking the VF PEs. An adapter driver and VF device drivers are informed of the error and initiate recovery.The hypervisor blocks access to the VF device drivers at critical points, allowing the adapter recovery to complete successfully.

[0010] According to features of the invention, a failure of a VF-PE causes only a failure of that one PE and is treated in isolation.

[0011] According to features of the invention, the VF device driver learns of the failure and initiates recovery, and the VF driver is blocked during the initial recovery steps, with the VF PEs remaining blocked until the adapter driver completes adapter recovery. The adapter driver unblocks the adapter PE, collects failure data, and initiates recovery and reinitialization, while the VF PE remains blocked. The adapter driver recovers the adapter, resets the VFs, and restores the adapter's previous configuration. The adapter driver grants permission to unblock the VF PEs, and the VF drivers begin recovery.

[0012] According to features of the invention, no coordination is required between the adapter driver and the VF device drivers, or between the VF device drivers. The VF device drivers can proceed independently of each other and complete the recovery independently of each other.

[0013] According to features of the invention, multiple isolation levels are provided. A first isolation level includes faults that are within the scope of a single VF. In this case, only the individual PE for the individual VF is blocked and recovered. A second isolation level includes at least one fault that is within the scope of the entire adapter. In this case, all PEs are blocked and recovered, including the adapter PE and each of the VF PEs. Brief description of the drawings

[0014] The present invention, together with the foregoing and other objects and advantages, will be best understood from the following detailed description of the preferred embodiments of the invention, which are illustrated in the drawings, in which: Fig. 1 and Fig. 2 illustrate, respectively, an exemplary computer system and an exemplary system for implementing enhanced error handling for a hardware I / O adapter, such as an SRIOV adapter, according to the preferred embodiment; Fig. 3, Fig. 4, Fig. 5 and Fig. 6 each show a flowchart illustrating exemplary operations for implementing extended error handling for the SRIOV adapter according to the preferred embodiment, and Fig. 7 is a block diagram illustrating a computer program product according to the preferred embodiment. Detailed description of the preferred embodiments

[0015] In the following detailed description of embodiments of the invention, reference is made to the accompanying drawings, which illustrate exemplary embodiments in which the invention may be practiced. It should be understood that other embodiments and structural changes may be made without departing from the scope of the invention.

[0016] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms of indefinite and definite articles such as "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, it is to be understood that the terms "comprises" and / or "comprising" used in this specification describe the presence of claimed features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups therefrom.

[0017] According to features of the invention, a method, system and computer program product are provided for implementing extended fault collection for a hardware I / O adapter such as a SRIOV adapter in a virtualized system.

[0018] Reference is made below to the drawings. Fig. 1 shows an exemplary computer system, generally designated by reference numeral 100, for implementing extended fault collection for a hardware I / O adapter 102 or SRIOV adapter 102, according to the preferred embodiment. The computer system 100 includes one or more processors 104 or central processing units (CPUs) 104 (one of which is shown) connected to the SRIOV adapter or hardware I / O adapter 102 via an I / O hub or processor host bridge 106. The processor host bridge (PHB) 106 provides diverse PE (partitionable endpoint) support, according to the preferred embodiment.

[0019] Computer system 100 includes memory 108 and one or more logical partitions (LPARs) 110 (one of which is shown) connected to processor 104 and processor host bridge 106 via system bus 111. Each operating system (OS) 112 resides in its own LPAR 110, with each LPAR being associated with a portion of a physical processor 104, an entire physical processor, or multiple physical processors of computer 100. A VF device driver 114 is provided along with logical partition (LPAR) 110. Each LPAR 110 is associated with a portion of memory 108. Computer system 100 includes a hypervisor 116 with a configuration mechanism 118. Hypervisor 116 is part of the system firmware and manages the allocation of resources to each operating system 112 and each LPAR 110.

[0020] As shown, a hardware management console (HMC) 120, used, for example, to manage system functions, including logical partition configuration and hardware virtualization, is connected to the hypervisor 116 via a service processor 122. The computer system 100 has a physical functions (PF) manager or PF auxiliary region 124 provided with the hypervisor 116. The PF auxiliary region 124 includes an adapter driver 128 for managing physical functions of the hardware I / O adapter 102. The hypervisor 116 uses the PF auxiliary region 124 to configure, for example, physical functions (PFs) and virtual functions (VFs) of the hardware I / O adapter 102 based on configuration data provided by a system administrator via the hardware management console 120.

[0021] For example, as shown, the hardware I / O adapter 102 includes a first physical function 130, a second physical function 132, a first port 134, and a second port 136. The hypervisor 116, using the PF auxiliary region 124, configures virtual functions based on the physical functions 130, 132 and maps virtual functions to one or more ports 134, 136 of the hardware I / O adapter 102.

[0022] For example, a first virtual function 140, instance 1, and the Mth instance of the first virtual function 142, where M is greater than 1, are assigned to the second port 136. As shown, a second virtual function 144, such as the first instance of the second virtual function 144 and the Pth instance of the second virtual function 146, where P is greater than 1, are assigned to the first port 134. As shown, multiple instances of an Nth virtual function, where N is greater than 2, such as the first instance of the Nth virtual function 148, are assigned to the first port 134, and the Qth instance of the Nth virtual function 150, where Q is greater than 1, is assigned to the second port 136.

[0023] Each instance of the first virtual function 140, 142, the second virtual function 144, 146, and the Nth virtual function 148, 150 is hosted by a physical function, such as one of the first physical function 130, the second physical function 132, and another physical function (not shown).

[0024] Each instance of the first virtual function 140, 142, the second virtual function 144, 146, and the Nth virtual function 148, 150 has a corresponding virtual function identifier (ID), shown as ID 152, ID 154, ID 156, ID 158, ID 160, and ID 162. Each virtual function identifier uniquely identifies a particular virtual function hosted by the hardware I / O adapter 102. For example, when a message (not shown) is directed to a particular virtual function, the message includes the identifier associated with the particular virtual function.

[0025] According to features of the invention, the PHB 106 with multiple PE support transforms a single physical adapter into multiple independent PCI endpoints. These endpoints can then be managed independently by the various partitions. Each VF 140, 142, 144, 146, 148, 150 is a unique PE. There is also a PE dedicated to the entire adapter 102. A failure of a VF PE causes only that one PE to fail and is advantageously treated in isolation. A failure of the adapter PE is propagated to the VF PEs associated with the adapter, causing them to fail as well.

[0026] A partitionable endpoint (PE) is a separately assignable I / O unit; that is, any part of an I / O subsystem that can be assigned a logical partition independent of any other PE. Each PE has independent scopes (addressing, faults, status, and the like) to provide PE-level fault isolation, detection, and recovery.

[0027] According to features of the invention, the hypervisor 116 provides basic support for managing the dedicated PEs. The adapter driver 128, which assigns dedicated functions and partitions to the PEs, provides sequential error recovery for the entire adapter 102, controls when the VFs 140, 142, 144, 146, 148, 150 are allowed to begin recovery, and manages the PFs 130, 132. The VF device driver 114 handles VF error recovery, for example, using the same sequence followed for a non-shared adapter.

[0028] Computer system 100 is shown in a simplified form sufficient for understanding the present invention. The illustrated computer system 100 is not intended to imply any architectural or functional limitations. The present invention may be used with various hardware implementations and systems, as well as various other internal hardware devices.

[0029] Fig. 2 shows another exemplary system, generally designated by reference numeral 200, for implementing distributed debug data collection and analysis for a hardware I / O adapter or SRIOV adapter 202, according to the preferred embodiment.

[0030] The system 200 includes a hypervisor 204 or other virtualization intermediary used to enable multiple logical partitions to access virtual functions provided by hardware, including the hardware I / O adapter 202. For example, as shown in Fig. 2, the hypervisor 204 uses a first physical function 220 of the hardware I / O adapter 202 to enable a first logical partition 206, a second logical partition 208, and an Nth logical partition 210 to access a plurality of virtual functions 212, 214, 216, 218 provided by the hardware I / O adapter 202. For example, the hypervisor 204 uses a first physical function 220 of the hardware I / O adapter 202 to provide a first instance of a first virtual function 212, a second instance of a first virtual function 214, and an Nth instance of a first virtual function 216 for the logical partitions 206, 208, 210. As shown, the hypervisor 204 uses a second physical function 222 of the hardware I / O adapter 202 to provide a second virtual function 218 for the logical partitions 206, 208, 210.

[0031] The physical functions 220, 222 advantageously include PCI functions that support single root I / O virtualization capabilities. Each of the virtual functions 212, 214, 216, 218 is associated with one of the physical functions 220, 222 and is configured to share one or more physical resources of the hardware I / O adapter 202.

[0032] Software functions or modules, such as a physical function auxiliary region (PF auxiliary region) 224 with an adapter driver 225, are provided to manage the physical functions 220, 222 and the virtual functions 212, 214, 216, 218 with the hypervisor 204. For example, a user can specify a specific configuration, and the hypervisor 204 uses the PF auxiliary region 224 to configure the virtual functions 212, 214, 216, 218 based on the physical functions 220, 222.

[0033] For example, during operation, the hypervisor 204 with the PF auxiliary region 224 activates the instances 212, 214, 216 of the first virtual function from the first physical function 220. The hypervisor 204 with the PF auxiliary region 224 activates the second virtual function 218 from the second physical function 222. The virtual functions 212, 214, 216, 218 are activated, for example, based on a configuration provided by a user. An operating system (not shown) and client applications (not shown) can run on each of the logical partitions 206, 208, 210.

[0034] As shown, the client applications executing on logical partitions 206, 208, 210 perform virtual input / output operations and have a corresponding device driver for directly managing an associated virtual function. For example, a first client application executing on first logical partition 206 may have a first client VF device driver 226, and a second client application executing on first logical partition 206 may have a second client VF device driver 228.

[0035] As shown, the first client VF device driver 226 accesses the first instance of the first virtual function 212. The second client VF device driver 228 accesses the second virtual function 218. A third client VF device driver 230 executing on the second logical partition 208 accesses the second instance of the first virtual function 214. An Nth client VF device driver 232 executing on the Nth logical partition 210 accesses the Nth instance of the first virtual function 216. An accessor mechanism 234 and a configuration mechanism 236 are provided with the hypervisor 204 to associate an accessed virtual function with a logical partition.The hypervisor 204 uses the access mechanism 234 to enable logical partitions such as the LPAR 206 to access a configuration area associated with one or more of the virtual functions 212, 214, 216, 218.

[0036] According to features of the invention, the hardware I / O adapter is partitioned into multiple endpoints, with each partitionable endpoint (PE) corresponding to a function and an adapter PE associated with the entire adapter. The endpoints are managed both independently for actions limited to a single function and as a group for actions across the adapter. A fault or failure of the adapter PE blocks the adapter PE and is propagated to the VF PEs associated with the adapter, thereby blocking the VF PEs. An adapter driver and VF device drivers are notified of the fault and initiate recovery. The hypervisor blocks access to the VF device drivers at critical points, enabling successful completion of adapter recovery.

[0037] According to features of the invention, the adapter unblocks its PE, collects error data, and initiates recovery / reinitialization, while the VF PEs remain blocked. The adapter driver recovers the adapter, resets the VFs, and restores the adapter's previous configuration. The adapter driver grants permission to unblock the VF PEs, and the VF drivers begin recovery.

[0038] System 200 is shown in a simplified form sufficient for understanding the present invention. The illustrated system 200 is not intended to imply any architectural or functional limitations. The present invention may be used with various hardware implementations and systems, as well as various other internal hardware devices.

[0039] According to features of the invention, extended error handling (EEH) is optionally implemented by the hardware detecting an error and blocking the adapter PE along with the downstream PEs. The same sequence can also be triggered by a number of other causes—this one serves only as an example.

[0040] The Fig. 3, Fig. 4, Fig. 5 and Fig. 6 illustrate exemplary operations of the processing and logic provided by the hypervisor 610 for implementing extended error handling (EEH) for a hardware I / O adapter in accordance with the preferred embodiment.

[0041] In Fig. 3, the hardware detects a failure, as shown in block 300, and blocks the adapter PE. As shown in block 302, other PEs associated with the adapter, such as VF PEs, are blocked. As shown in block 304, the hardware informs the hypervisor about the blocked PEs. The hypervisor informs PE owners, including the adapter driver and VF device driver, about the blocked PEs, as shown in block 306. The adapter driver and VF device driver begin recovery asynchronously, as shown in block 308.

[0042] In Fig. 4, the adapter driver receives notification of the error, as shown in block 400. The adapter driver begins EEH recovery, as shown in block 402. The adapter driver unblocks the adapter PE, but not other PEs, collects error data, and begins recovery, as shown in block 404.

[0043] As shown in block 406, the adapter driver recovers the adapter. This may involve resetting the entire adapter, which also resets the VFs. The adapter is then restored to the preset state at block 406. As shown in block 408, the adapter driver restores the previous configuration to the adapter. This is the same configuration as before, since the VFs must be restored similarly to what was previously present, for example, with the same PCI BAR spacing or VF BAR registers, and the like. The adapter driver then logs the error and transmits a log ID to the hypervisor, as shown in block 410. As shown in block 412, the adapter driver grants the hypervisor permission to unblock VF PEs and resumes normal operation.

[0044] In Fig. 5, the VF device driver receives notification of the error, as shown at block 500. The VF device driver begins an EEH recovery, as shown at block 502. The VF device driver loops in its attempt to unblock the PEs, and the attempt is unsuccessful until the adapter driver completes the recovery, as shown at block 504. The adapter driver completes the recovery, as shown at block 506. The VF device driver unblocks the PE, retrieves error data, and begins a recovery, as shown at block 508. The VF device driver completes the recovery and logs error data, as shown at block 510. Normal VF and I / O operations then resume, as shown at block 512.

[0045] According to features of the invention, an extended error handling (EEH) optionally includes two isolation levels, such as in Fig. 3, Fig. 4 and Fig. 5. The first isolation level includes faults that are within the scope of a single VF. In this case, only the single PE for the single VF is blocked and recovered. The second isolation level includes faults that are within the scope of the entire adapter. In this case, all PEs are blocked and recovered, including the adapter PE and each of the VF PEs.

[0046] According to features of the invention, enhanced error handling (EEH) optionally includes additional intermediate groupings. This adds additional levels to the hierarchy. The advantage of this is more coordinated recovery. The recovery action is less disruptive, and fewer VFs are affected. For example, some faults may be in the scope of a single physical port. In this case, recovery may involve blocking the set of PEs associated with the VFs that use that physical port. In another example, faults may be in the scope of a single I / O protocol. For example, for a converged network adapter (CNA) that implements both Network Interface Controller (NIC) and Fibre Channel over Ethernet (FCoE), a fault may affect only FCoE.In this case, recovery may involve blocking the set of PEs associated with VFs using that protocol, while leaving PEs for VFs running other protocols unblocked.

[0047] According to features of the invention, extended fault handling (EEH) optionally includes multiple and potentially overlapping groups of PEs. Continuing the above examples, there may be groupings for all PEs on a single port, all FCoE PEs on that port, and also all NIC PEs on the same port. Furthermore, there may be further groupings of all FCoE PEs across all ports of the adapter. The exact details of the groupings used are determined by the isolation level provided by the adapter vendor for recovery. Allowing different groupings by the adapter driver enables maximum fault isolation with the minimum number of affected VF PEs. It should be noted that recovery by the VF driver is independent of the groupings used, since the VF driver only acts on a single VF.

[0048] Fig. 6 shows an exemplary device, generally designated by reference numeral 600, to illustrate an exemplary protocol flow of a higher-level error protocol for error protocols triggered by the adapter or a PF. As in Fig. 6, the device 600 includes a plurality of LPARs 1-N, 602 with an AIX, Linux, or IBM i operating system and a VF driver 606. A hypervisor 610 includes a PF auxiliary region 612 with an error log 614 and an adapter driver 616, a log ID 618, and error logging 619. An SRIOV adapter 620 includes a plurality of PFs 622 and a plurality of VFs 624. A hardware management console (HMC) 630 is connected to the hypervisor 610 via a field service processor (FSP) 632.

[0049] In Fig. 6, a first step is shown by a solid line labeled 1 for a firmware error log flow. A PF auxiliary region begins the process of creating a platform error log by issuing an AMC (Application Management Console) request to the hypervisor to log an error. The PF auxiliary region 612 provides the hypervisor's 610 error logging support 619 (SRC), severity, action markers, and other data. Once the error log 614 is created, the log ID 618 is returned to the PF auxiliary region 612.

[0050] A second step is shown by a line labeled 2: The error logging support 619 of the hypervisor 610 sends the log to the FSP 632 for inclusion in the log flow. A third step is shown by a line labeled 3: The FSP 632 sends the log to the HMC 630. A fourth step is shown by a line labeled 4: The FSP 632 sends the log back to the hypervisor 610 for a full broadcast. A next, fifth step is shown by lines labeled 5: The error logging support 619 of the hypervisor 610 sends the log to all active partitions 602.An optional sixth step is shown by a dashed line labeled 6 for a Resource Monitoring and Control (RMC) connection, where the HMC 630 can pull the platform error log from platform operating systems using the RMC connection. This provides a redundant path for receiving platform error logs. An optional seventh step is shown by a dashed line labeled 7 for a Hypervisor Call (HCALL) interface: Partition 602 can query the error log ID 618 from the hypervisor 610.

[0051] In Fig. 7 illustrates a product or computer program product 700 of the invention. The computer program product 700 is physically embodied on a non-transitory computer-readable storage medium having a recording medium 702, such as a floppy disk, a high-capacity read-only memory in the form of an optically read compact disk or CD-ROM, a tape, or other similar computer program product. The recording medium 702 stores the program means 704, 706, 708, and 710 on the medium 702 to implement the methods for implementing improved error collection for an I / O adapter, such as a SRIOV adapter of a preferred embodiment, in the system 100 of Fig. 1 or the System 200 Fig. 2 to be executed.

[0052] A sequence of program instructions defined by the recorded program means 704, 706, 708 and 710 or a logical arrangement of one or more interrelated modules directs the computer system 700 to implement an extended error collection for the I / O adapter.

[0053] Although the present invention has been described with reference to the details of the embodiments of the invention shown in the figures, these details are not intended to limit the scope of the invention as claimed in the appended claims.

Claims

[1] A method for implementing extended fault collection for an input / output adapter (I / O adapter) in a computer system, wherein the I / O adapter is partitioned into a plurality of partitionable endpoints, each partitionable endpoint (PE) corresponding to a function, and wherein the method comprises an adapter PE associated with the I / O adapter and a plurality of virtual function PEs, and wherein the method comprises: in response to an adapter failure, blocking the adapter PE; Blocking the VF PEs associated with the adapter in response to a blocking of the adapter PE; Informing an adapter driver and VF device drivers about the error and initiating a recovery by the adapter driver and VF device drivers, and Blocking access to the VF device drivers, allowing the adapter recovery to complete successfully. [2] The method of claim 1, wherein the I / O adapter comprises a single-root input / output virtualization (SRIOV) adapter. [3] The method of claim 1, comprising the adapter driver unblocking the adapter PE, collecting error data, and starting recovery and reinitialization, wherein the VF PEs remain blocked. [4] The method of claim 3, comprising restoring the adapter by the adapter driver and restoring a configuration of the adapter. [5] The method of claim 4, comprising granting permission by the adapter driver to unblock the VF PEs, and the VF device drivers commencing recovery. [6] The method of claim 1, comprising the VF device drivers unblocking the VF PEs in response to receiving permission to unblock the VF PEs, and the VF device drivers commencing recovery. [7] The method of claim 6, comprising collecting error data by each VF device driver. [8] The method of claim 6, comprising completing the recovery and logging error data by each VF device driver. [9] The method of claim 8, comprising resuming normal VF and I / O operations by the VF device drivers. [10] The method of claim 1, comprising notifying a system hypervisor that the adapter PE and the VF PEs are blocked, wherein the system hypervisor informs the adapter driver and the VF device drivers of the error. [11] The method of claim 10, comprising asynchronously initiating recovery by the adapter driver and the VF device drivers in response to being informed of the failure by the system hypervisor. [12] A system for implementing extended fault collection for an input / output adapter (I / O adapter) in a computer system, wherein the I / O adapter is partitioned into a plurality of partitionable endpoints, each partitionable endpoint (PE) corresponding to a function, and wherein the method includes an adapter PE associated with the I / O adapter and a plurality of virtual function PEs (VF PEs), and wherein the system comprises: a processor, a hypervisor that manages functions associated with the hardware I / O adapter; where the processor uses the hypervisor to perform the following steps: in response to an adapter failure, blocking the adapter PE; Blocking the VF PEs associated with the adapter in response to a blocking of the adapter PE; Informing an adapter driver and VF device drivers about the error and initiating a recovery by the adapter driver and VF device drivers, and Blocking access to the VF device drivers, allowing the adapter recovery to complete successfully. [13] The system of claim 12, comprising unblocking the VF PEs by the VF device drivers in response to receiving permission to unblock the VF PEs, wherein the VF device drivers begin recovery. [14] The system of claim 13, comprising completion of recovery and logging of error data by each VF device driver. [15] The system of claim 14, comprising resuming normal VF and I / O operations by the VF device drivers in response to completion of the recovery.

Images