IO device error handling methods, hosts, devices, media and products

Abstract

The embodiment of the present application provides an error handling method, host machine, device, medium and product for an IO device, and relates to the field of computer technology. The method includes: in response to an error interrupt request for a target error, obtaining target error information, the error interrupt request is triggered when a physical IO device finds a target error, the physical IO device is virtualized into multiple virtualized IO devices, and the multiple virtualized IO devices are assigned to multiple virtual machines; based on the target error information, stopping the operation of a target virtual machine among the multiple virtual machines, wherein the target error information includes associated information associated with the impact range of the target error, and the target virtual machine is a virtual machine within the impact range of the target error. The technical solution of the embodiment of the present application can realize error isolation on demand, thereby improving the stability and reliability of the system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an error handling method for an IO device, a host machine, an electronic device, a computer-readable storage medium, and a computer program product, which can be applied to the field of IO virtualization technology. Background Art

[0002] With the widespread application of input-output (IO) virtualization technology, especially in the field of high-performance computing and data centers, the virtualization of IO devices has become one of the key technologies for achieving efficient resource utilization. Single Root Input Output Virtualization (SR-IOV) improves the utilization efficiency of hardware resources by dividing a physical IO device into multiple virtual functions (VFs), allowing multiple virtual machines to share the same physical IO device. In this architecture, each VF can be independently passed through to the virtual machine. This pass-through method brings nearly native hardware access performance, but also introduces a series of error handling problems. For example, when an error occurs in a VF of a physical IO device, all virtual machines that pass through the VF will be stopped, resulting in performance degradation and resource waste of the entire system. Summary of the invention

[0003] Embodiments of the present application provide an IO device error method, a host machine, an electronic device, a computer-readable storage medium, and a computer program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present application provides an error handling method for an IO device, comprising: obtaining target error information in response to an error interrupt request for a target error, wherein the error interrupt request is triggered when a physical IO device discovers a target error, and the physical IO device is virtualized into multiple virtualized IO devices, and the multiple virtualized IO devices are assigned to multiple virtual machines; based on the target error information, stopping the operation of a target virtual machine among the multiple virtual machines, wherein the target error information includes associated information associated with a scope of influence of the target error, and the target virtual machine is a virtual machine within the scope of influence of the target error.

[0005] In a second aspect, an embodiment of the present application provides a host machine, comprising: a physical IO device, wherein the physical IO device is virtualized into multiple virtualized IO devices, and the multiple virtualized IO devices are assigned to multiple virtual machines; a processor, wherein the processor carries the multiple virtual machines, and the processor is used to implement any method of the embodiments of the present application when an erroneous interrupt request is received.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor implements any method of the embodiment of the present application when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any one of the embodiments of the present application is implemented.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any method of the embodiments of the present application when executed by a processor.

[0009] According to the technical solution of the embodiment of the present application, when an error occurs in a virtualized IO device on a physical IO device (i.e., a target error), target error information is obtained. Since the target error information contains associated information associated with the impact scope of the target error, the operating system kernel can stop the operation of the corresponding target virtual machine according to the impact scope of the target error, thereby realizing on-demand error isolation, thereby improving the stability and reliability of the system.

[0010] Optionally, the virtual machines that need to be stopped are determined based on the error type of the target error. For example, error isolation is performed when the error type is an isolatable error type, and only the virtual machines affected by the error are stopped, so that the processing logic of the requester / receiver in the device or system management software can recover from the error without resetting link components and interfering with other ongoing transactions, thereby achieving more accurate and efficient error isolation, which can further improve the reliability and availability of the virtualization system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0012] Figure 1 , Figure 2 and Figure 3 The IO virtualization method in the related art is shown;

[0013] Figure 4 A schematic diagram showing the structure of a host machine provided in an embodiment of the present application is shown;

[0014] Figure 5 A schematic diagram of a system architecture of a host machine provided in an embodiment of the present application is shown;

[0015] Figure 6 A flowchart showing an error handling method for an IO device according to an embodiment of the present application is shown;

[0016] Figure 7 An example diagram showing an application of the error handling method of an IO device according to an embodiment of the present application;

[0017] Figure 8 A block diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0019] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0020] IO virtualization includes managing the routing of IO requests between virtual devices and shared physical hardware. Currently, there are three main ways to implement IO virtualization: IO full virtualization, IO pass-through, and hardware-assisted virtualization.

[0021] In full virtualization, the Virtual Machine Monitor (VMM) simulates multiple virtualized IO devices for use by the Virtual Machine (VM). Figure 1As shown, the VMM simulates multiple virtualized IO devices to reuse the same physical IO device (for example, a network interface card (NIC)). The virtualized IO devices simulated by the VMM are allocated to virtual machines VM1 and VM2 for use, thereby allowing multiple virtual machines (VM1 and VM2) to share limited physical peripheral resources. The VMM intercepts the access requests of the client (Guest) operating system to the data plane and control plane of the physical IO device and performs simulated operations. This method will cause frequent context switching of the operating system, which will lead to lower performance of the operating system.

[0022] IO direct transmission technology can also be called IO transparent transmission technology. Figure 2 As shown in the figure, in the IO direct pass-through technology, the physical IO device (such as NIC) can be directly allocated to virtual machine 1 for use. This method requires the hardware platform to support IO transparent transmission technology, which can achieve near-native performance and low CPU (Central Processing Unit) overhead. However, the disadvantage of IO direct pass-through is that multiple virtual machines cannot share the same physical IO device, that is, Figure 2 Virtual Machine 2 in cannot access physical IO devices (such as NIC) at the same time.

[0023] Common hardware-assisted virtualization technologies include SR-IOV, such as Figure 3 As shown, a physical IO device (such as NIC) is virtualized into multiple VFs (including virtualized IO device 1 and virtualized IO device 2). Each VF can be used as a virtualized IO device, with independent bus configuration space and device information (including bus number, device number and function number), which can be directly allocated to different virtual machines (virtual machine 1 and virtual machine 2), thereby providing high-performance resource sharing. For example, SR-IOV technology allows a physical IO device to support up to 256 VFs while minimizing the hardware cost of each function.

[0024] Taking a physical IO device as a physical peripheral component interconnect (PCI) device as an example, the following table shows the comparison results of three IO virtualization methods for the PCI device.

[0025]

[0026] In order to reduce CPU utilization and reduce the CPU consumption of the host (Host) on network, storage and security functions, some computing tasks can be offloaded through dedicated hardware. Today, SR-IOV has been widely adopted in high-performance data transmission scenarios, especially in public cloud environments, such as non-volatile memory express (NVMe) solid-state drives (SSDs), high-performance network cards, graphics processing units (GPUs), hardware accelerators (such as Quick Assist Technology (QAT)), etc. Through SR-IOV technology, VF is passed directly to the virtual machine to support enhanced networking, which can increase bandwidth by 20% and reduce latency by 50%.

[0027] In public clouds, the virtual machine downtime rate is a key indicator in the service level agreement (SLA), and error isolation has a significant impact on reducing the downtime rate. However, in SR-IOV technology, when a VF of a physical IO device fails, all virtual machines connected to the VF will be stopped, causing multiple virtual machines with irrelevant errors to stop running, failing to achieve error isolation, and affecting system stability and SLA compliance rate.

[0028] In view of this, the embodiments of the present application aim to provide an IO device error handling method, a host machine, an electronic device, a computer-readable storage medium and a computer program product, which can realize on-demand error isolation, thereby improving the stability and reliability of the system.

[0029] The following is a detailed introduction. For ease of understanding, first combine Figure 4 and Figure 5 The system architecture of the technical solution of the embodiment of the present application is introduced. Figure 4 A schematic diagram of the hardware architecture of a host machine provided in an embodiment of the present application is shown. Figure 5 A schematic diagram of a software architecture of a host machine provided by an embodiment of the present application is shown. Specifically, the hardware layer structure of the host machine may include a processor 201 , a bus system 202 , and a memory 203 .

[0030] The processor 201 is, for example, a CPU; the memory 203 is, for example, a random access memory (RAM), which provides a physical address space of the host machine. The number of the processor 201 and the memory 203 can be one or more, and they are interconnected through the bus system 202 to complete mutual communication.

[0031] When the host machine is started, the processor 201 first reads the startup program to load the host operating system. The host operating system runs in the host kernel space. In the virtualization scenario, the user can start one or more virtual machines 204 in the host user space, including installing the guest operating system on the virtual machine 204. In virtualization technology, Guest can be used to represent a virtual machine, such as Figure 5 As shown, Guest1 and Guest2 represent virtual machine 1, virtual machine 2, etc., respectively. Furthermore, multiple virtual machine managers are run in the Host user space. The virtual machine manager is, for example, a Quick Emulator (QEMU), which can simulate multiple processor architectures and accelerate the operation of virtual machines through hardware-assisted virtualization technology. For example, manager 1 (QEMU1) is responsible for managing virtual machine 1 (Guest1), and manager 2 (QEMU2) is responsible for managing virtual machine 2 (Guest2).

[0032] The bus system 202 is, for example, a PCI bus system, a peripheral component interconnect express (PCIe) bus system, etc. Taking the PCIe bus system as an example, the system architecture of the bus system 202 generally includes a root complex, a bus, an endpoint, and a switch. The root complex is the root node of the entire bus system 202. On the root complex, there are many root ports (RP), i.e., bus root ports, such as bus root port (RP) 222A and bus root port (RP) 222B. Each RP can be connected to one or more physical IO devices, and a physical IO device is a terminal. For example, RP222A is connected to two physical IO devices, i.e., terminal 221A and terminal 221B, through a switch, while RP222B is connected to a physical IO device, i.e., terminal 221C.

[0033] In the PCIe bus system, the physical IO device is the PCIe device. The processor 201 is connected to the root complex (that is, connected to each RP), and finally connected to all physical IO devices. The RP is responsible for managing and controlling the physical IO devices connected to it. It can pass the IO request of the physical IO device to the processor 201 and send the instructions of the processor 201 to the physical IO device.

[0034] In SR-IOV technology, physical IO devices are virtualized into multiple VFs. Each VF, as a virtualized IO device, has an independent bus configuration space and device information (including bus number, device number and function number, referred to as BDF), which can be directly allocated to virtual machines for use. In other words, the virtualized IO device (VF) can provide virtual machines with access to physical IO device resources, allowing multiple virtual machines to share a physical IO device. Virtual machines can directly pass through VFs, that is, virtual machines can directly interact with VFs to implement corresponding IO operations. For example, Figure 5 As shown, the physical IO device is virtualized into VF1, VF2, ... Among them, VF1 provides Guest1 with access rights to physical IO device resources, that is, Guest1 can directly connect to VF1 to implement IO operations; VF2 provides Guest2 with access rights to physical IO device resources, that is, Guest2 can directly connect to VF2 to implement IO operations; ...

[0035] For example, Figure 5 As shown in the figure, when the physical IO device detects that VF1 has an error, the error information (target error information) of the error will be recorded in the error information register. The physical IO device reports the error message to its upstream port through the error message notification mechanism, and finally routes it to the RP to which the physical IO device is connected. For example, Figure 4 As shown, if the physical IO device is terminal 221C, the uplink port of the physical IO device is RP222B, and the error message will be directly reported to RP222B; if the physical IO device is terminal 221A, the uplink port of the physical IO device is port 223A of the switch, and the switch routes the error message to RP222A to which the physical IO device is connected; if the physical IO device is terminal 221B, the uplink port of the physical IO device is port 223B of the switch, and the switch routes the error message to RP222A to which the physical IO device is connected.

[0036] like Figure 5As shown, the RP connected to the physical IO device triggers an interrupt after receiving an error message, that is, sends an error interrupt signal to the operating system kernel of the processor; the error handling driver of the operating system kernel responds to the error interrupt signal and obtains the target error information, for example, reading the target error information from the error information register; the error handling driver stops the operation of the target virtual machine Guest1 according to the impact scope of the target error, for example, by sending an error notification to QEMU1, so that QEMU1 stops the operation of Guest1, or the error handling driver stops the operation of all virtual machines under the upstream port according to the impact scope of the target error, for example, sends an error notification to QEMU1, QEMU2..., so that QEMU1 stops the operation of Guest1, QEMU2 stops the operation of Guest2...

[0037] That is to say, in the embodiment of the present application, when an error occurs in a virtualized IO device on a physical IO device, the operating system kernel first obtains the error information of the error. Since the error information contains associated information associated with the impact range of the target error, the operating system kernel can stop the operation of the corresponding target virtual machine according to the impact range of the target error. The target virtual machine can be one, multiple, or all, depending on the impact range of the target error. Compared with the related art, as long as an error occurs in a virtualized IO device on a physical IO device, the operating system kernel will traverse each virtualized IO device under the upstream port of the virtualized IO device (that is, the upstream port of the current physical IO device) and stop the operation of each corresponding virtual machine. The technical solution of the embodiment of the present application will stop the operation of the corresponding virtual machine as needed according to the error information of the error, thereby realizing error isolation on demand.

[0038] The technical solution of the present application is described in detail with specific embodiments below. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0039] Figure 6 The flowchart of the error handling method of the IO device in the embodiment of the present application is shown, and the error handling method can be applied to the operating system kernel. Figure 6 As shown, the error handling method may include step S601 and step S602.

[0040] Step S601: In response to an error interrupt request for a target error, target error information is obtained, wherein the error interrupt request is triggered when a physical IO device finds a target error, the physical IO device is virtualized into multiple virtualized IO devices, and the multiple virtualized IO devices are allocated to multiple virtual machines.

[0041] In an embodiment of the present application, the physical IO device may be a PCI device or a PCIe device. The PCIe device includes but is not limited to a graphics processing unit (GPU), a NIC, a sound card, a solid state drive (SSD), etc.

[0042] The physical IO device is virtualized into multiple VFs. Each VF, as a virtualized IO device, can provide access rights to the physical IO device resources for one or more virtual machines (usually one), thereby allowing multiple virtual machines to share the physical IO device. For example, a VF can be assigned to each virtual machine, that is, multiple VFs have a one-to-one mapping relationship with multiple virtual machines. One VF belongs to one virtual machine, and provides the virtual machine with access rights to the physical IO device resources, so that the virtual machine can directly access the VF, thereby allowing multiple virtual machines to share the resources of the physical IO device and implement corresponding IO operations. For example, Figure 5 As shown, the physical IO device is virtualized into VF1, VF2, ..., VF1 belongs to Guest1, and provides Guest1 with access rights to the physical IO device resources, so that Guest1 can directly connect to VF1 to implement IO operations; VF2 belongs to Guest2, and provides Guest2 with access rights to the physical IO device resources, so that Guest2 can directly connect to VF2 to implement IO operations; ...

[0043] During system operation, if an error occurs in a virtualized IO device (i.e., the target virtualized IO device), the physical IO device can detect the error, i.e., discover the target error, and then trigger an error interrupt signal. Taking the physical IO device as a PCIe device as an example, the PCIe protocol specifies the Advanced Error Reporting (AER) mechanism. If an error occurs in the target virtualized IO device, i.e., an AER error, the PCIe device reports the error message (ERRMessage) to its upstream port through the AER mechanism, and finally routes it to the PCIe bus root port connected to it. The PCIe bus root port triggers the AER interrupt and sends an error interrupt signal to the operating system kernel of the processor. If the operating system kernel receives an error interrupt request, it obtains the target error information. For example, the interrupt processing function registered in the AER error handling driver of the operating system kernel will respond to the error interrupt signal and obtain the target error information.

[0044] In one example, the error handling driver can read the target error information from the error information register. In other examples, the physical IO device can also send the target error information to the operating system kernel of the processor through the bus, and the method for obtaining the target error information in the embodiment of the present application is not specifically limited.

[0045] Step S602: based on the target error information, stopping the operation of a target virtual machine among the multiple virtual machines, wherein the target error information includes associated information associated with the impact scope of the target error, and the target virtual machine is a virtual machine within the impact scope of the target error.

[0046] In the embodiment of the present application, the associated information can determine the impact scope of the target error, and then determine the target virtual machine that needs to be stopped based on the impact scope.

[0047] In one example, the associated information may be information about the error type of the target error. Based on the error type of the target error, the impact scope of the target error may be determined, and the target virtual machine that needs to stop running is the virtual machine within the impact scope. For example, if the error type of the target error is a non-fatal error, it means that the target error only affects the virtual machine corresponding to the target virtualized IO device, that is, the target virtual machine is a virtual machine with IO access rights to the target virtualized IO device. For another example, if the error type of the target error is a fatal error, it means that the target error will affect all virtual machines that share the physical IO device, that is, the target virtual machine is the virtual machine to which each VF of the current physical IO device belongs. In other examples, the error information may be identification information of the virtual machine affected by the target error, that is, the target virtual machine can be directly determined through the target error information, and the embodiments of the present application are not limited to this.

[0048] Exemplarily, the operating system kernel sends an error notification to the manager of the target virtual machine, thereby causing the manager of the target virtual machine to stop the operation of the target virtual machine. Figure 5 As shown, if the target virtual machine is Guest1, the operating system kernel sends an error notification to QEMU1, and QEMU1 stops the operation of Guest1; if the target virtual machine is Guest1, Guest2..., error notifications are sent to QEMU1, QEMU2... respectively, and QEMU1 stops the operation of Guest1, and QEMU2 stops the operation of Guest2; ...

[0049] In the related art, any error on a physical IO device will cause all virtual machines sharing its resources to stop running. According to the technical solution of the embodiment of the present application, when an error occurs in a virtualized IO device on the physical IO device (i.e., a target error), the operating system kernel first obtains the target error information. Since the target error information contains associated information associated with the impact scope of the target error, the operation of the corresponding target virtual machine can be stopped according to the impact scope of the target error, thereby realizing on-demand error isolation, thereby improving the stability and reliability of the system.

[0050] In one embodiment, the associated information is information about the error type of the target error, and the target error information also includes device information of the target virtualized IO device. The target virtualized IO device is the virtualized IO device where the target error occurs. In step S601, obtaining the target error information may include: reading the device information of the target virtualized IO device from a first register, wherein the first register is a register of the bus root port that triggers the error interrupt request; and reading the error type information of the target error from a second register, wherein the second register is a register of the target virtualized IO device.

[0051] Exemplarily, when the physical IO device detects that an error has occurred in the target virtualized IO device (i.e., a target error), it will record the error type information of the target error in the register of the target virtualized IO device (i.e., the second register), and report the error message to the RP to which the physical IO device is connected through the error message notification mechanism. After receiving the reported error message, the RP will record the error source information (i.e., the device information of the target virtualized IO device) in its register (i.e., the first register), and trigger an error interrupt request.

[0052] Taking the AER mechanism of PCIe devices as an example, the first register and the second register both belong to the AER error register. The address offset of the AER error register is specified in the PCIe protocol. The interrupt processing function registered in the AER error processing driver of the operating system kernel will access the first register based on the address offset of the first register to read the device information, and access the second register based on the address offset of the second register to read the error type information of the target error.

[0053] By reading the first register and the second register, the target error information can be accurately identified, helping the system to quickly determine the target virtual machine, thereby reducing the diagnosis time.

[0054] In one implementation, the associated information is information about the error type of the target error. By parsing the associated information, the error type of the target error can be determined, and then the target virtual machine can be determined.

[0055] The associated information is information about the error type of the target error. By parsing the associated information, the error type of the target error can be obtained. The error type of the target error is usually associated with the scope of influence of the target error. For example, if the error type of the target error is a non-fatal error, it means that the target error only affects the virtual machine corresponding to the target virtualized IO device, and the target virtual machine can be determined to be the virtual machine corresponding to the target virtualized IO device; if the error type of the target error is a fatal error, it means that the target error will affect all virtual machines sharing the current physical IO device, and the target virtual machine can be determined to be all virtual machines sharing the current physical IO device.

[0056] In one implementation, when the error type of the target error is an isolatable error type, the target virtual machine is a virtual machine to which the target virtualized IO device belongs, and the target virtualized IO device is a virtualized IO device where the target error occurs.

[0057] The isolable error type indicates that the impact of the target error is limited, usually only affecting the virtual machine corresponding to the target virtualized IO device where the error occurs. Therefore, it is only necessary to stop the operation of the virtual machine to which the target virtualized IO device belongs, that is, only isolate the error of this virtual machine, so that other virtual machines can work normally and are not affected by the target error.

[0058] Exemplarily, the isolable error types include transactional error types. Transactional errors are, for example, non-fatal errors, which usually only affect a specific transaction or a part of a device. After the error occurs, although the specific transaction is unreliable, the link functions normally in other aspects, and other parts can continue to work without affecting the entire physical layer or link layer. Therefore, transactional errors belong to isolable error types, and error isolation can be performed for transactional errors, so that the processing logic of the requester / receiver in the device or system management software can recover from the error without resetting the link component and interfering with other ongoing transactions, so that devices unrelated to the transactional error will not be affected by the error.

[0059] In one embodiment, the target error information also includes device information of the target virtualized IO device. In step S602, based on the target error information, stopping the operation of the target virtual machine among the multiple virtual machines may include: in response to the error type of the target error being an isolatable error type, determining the target manager corresponding to the device information according to the mapping relationship between the device information of the multiple virtualized IO devices and the managers of the multiple virtual machines; and sending an error notification to the target manager, the error notification being used to notify the target manager to stop the operation of the target virtual machine it manages.

[0060] The device information of the target virtualized IO device is the identification code of the virtualized IO device, such as BDF. Taking the physical IO device as a PCIe device as an example, each VF (i.e., virtualized IO device) is assigned a unique BDF. When an error occurs in the target virtualized IO device, the target error information containing its BDF information will be recorded. By parsing the target error information, the BDF of the target virtualized IO device can be obtained.

[0061] The operating system kernel can determine the manager of the target virtual machine (i.e., the target manager) according to the mapping relationship through the BDF of the target virtualized IO device. For example, the manager of each virtual machine usually also has its own identifier. Therefore, in a computer device, the mapping relationship between multiple virtualized IO devices and managers of multiple virtual machines can be expressed as a mapping relationship between the BDFs of multiple virtualized IO devices and the identifiers of managers of multiple virtual machines. The operating system kernel searches for the identifier of the target manager through the BDF of the target virtualized IO device, and then sends an error notification to the target manager, informing the target manager to stop the operation of the target virtual machine. For example, Figure 5 As shown, if the target virtualized IO device is VF1, the operating system kernel sends an error notification to QEMU1, and QEMU1 stops the operation of Guest1.

[0062] Taking the AER mechanism of PCIe devices as an example, the AER error handling driver of the operating system kernel obtains the BDF of the target virtualized IO device and determines the target manager according to the mapping relationship. Then, by calling the error handling function (such as pci_error_handlers) of the virtualized function input and output (VFIO) driver of the operating system kernel, the target manager is notified of an event (i.e., an error notification) to stop the operation of the target virtual machine. For example, Figure 5 As shown, if the target virtualized IO device is VF1, the AER error handling driver calls the error handling function of the VFIO driver, sends an error notification to QEMU1 through an event notification (eventfd), and notifies QEMU1 to stop the operation of Guest1; QEMU1 sends a virtual machine stop command (vm_stop) to Guest1 by calling its registered error response function (such as vfio_err_notifier_handler) to stop the operation of Guest1.

[0063] Based on this, when an isolable error type occurs in the target virtualized IO device, only the virtual machine to which the target virtualized IO device belongs can be used as the target virtual machine, and the target manager can be found based on the mapping relationship between multiple virtualized IO devices and managers of multiple virtual machines. The target manager then stops the operation of the target virtual machine, thereby enabling the management logic in the virtual machine scenario to be used to achieve fast and effective error isolation without adding new management logic for error isolation of the virtual machine.

[0064] In one embodiment, when the error type of the target error is a non-isolated error type, the target virtual machine is a plurality of virtual machines, that is, all virtual machines that share the current physical IO device. The non-isolated error type indicates that the impact scope of the target error may cover all virtual machines that share the current physical IO device, so it is necessary to stop each virtual machine that shares the current physical IO device, so as to avoid service errors in the system.

[0065] Exemplarily, the non-isolated error type includes a link layer error or a physical layer error. Link layer errors or physical layer errors are, for example, fatal errors, which cause a specific link and related hardware to be unreliable and in an unrecoverable error state. Fatal errors require resetting components on the link to restore reliable operation. Therefore, link layer errors or physical layer errors belong to non-isolated error types, that is, it is not possible to stop the operation of one or some virtual machines alone, and it is necessary to stop the operation of all virtual machines sharing the current physical IO device.

[0066] In one embodiment, the target error information also includes device information of a target virtualized IO device, where the target virtualized IO device is a virtualized IO device where a target error occurs, and the target virtualized IO device is connected to an uplink port. In step S602, based on the target error information, stopping the operation of a target virtual machine among multiple virtual machines may include: in response to an error type of the target error being a non-isolated error type, traversing each virtualized IO device connected to the uplink port, and sending error notifications to managers of virtual machines corresponding to each virtualized IO device, respectively, where the error notifications are used to notify the corresponding manager to stop the operation of the target virtual machine it manages.

[0067] For example, Figure 4 As shown, if the physical IO device is terminal 221C, the uplink port of the physical IO device and its corresponding target virtualized IO device is RP222B; if the physical IO device is terminal 221A, the uplink port of the physical IO device and its corresponding target virtualized IO device is port 223A of the switch; if the physical IO device is terminal 221B, the uplink port of the physical IO device and its corresponding target virtualized IO device is port 223B of the switch.

[0068] Exemplarily, the operating system kernel can obtain the BDF of the target virtualized IO device by parsing the target error information, and then traverse each virtualized IO device connected to the upstream port of the target virtualized IO device, determine the corresponding manager of each virtual machine according to the mapping relationship, and send an error notification to the manager of each virtual machine, informing each manager to control the virtual machine it manages to stop running.

[0069] Taking the AER mechanism of PCIe devices as an example, the AER error handling driver of the operating system kernel traverses the upstream ports connected to the target virtualized IO device, obtains the BDF of each PCIe device on the bus, and determines the identifier of the manager of each virtual machine according to the mapping relationship, calls the error handling function of the VFIO driver, and sends an error notification to each manager through the event notification of each manager. For example, Figure 5 As shown, the AER error handling driver calls the error handling function of the VFIO driver, and sends error notifications to QEMU1, QEMU2, etc. through event notifications for each manager; QEMU1 sends a virtual machine stop command to Guest1 by calling its registered error response function to stop the operation of Guest1; QEMU2 sends a virtual machine stop command to Guest2 by calling its registered error response function to stop the operation of Guest2; ...

[0070] The operating system kernel does not know the information of the physical IO device, so it can traverse its connected ports based on the device information of the target virtualized IO device, and then send error notifications to the corresponding virtual machine managers to stop the operation of the virtual machine. In this way, when a non-isolated error occurs, the operation of each virtual machine on the link can be stopped without changing the error handling mechanism of the operating system kernel, so as to facilitate the subsequent resetting of the components on the link to restore reliable operation.

[0071] In an application example, the operating system kernel obtains target error information by reading a first register and a second register, and parses the target error information to determine an error type of the target error and a BDF of a target virtualized IO device; when the error type of the target error is an isolatable error type such as a transactional error (such as a non-fatal error), the target virtual machine is determined to be a virtual machine with IO access rights to the target virtualized IO device, and according to the BDF of the target virtualized IO device and the mapping relationship between each virtualized IO device and the manager of each virtual machine, the target manager corresponding to the target virtualized IO device is determined, and an error notification is sent to the target manager to notify the target manager to stop the operation of the target virtual machine; when the error type of the target error is a non-isolatable error type such as a link layer error or a physical layer error (such as a fatal error), the target virtual machine is determined to be each virtual machine that shares the current physical IO device, and according to the mapping relationship between each virtualized IO device and the manager of each virtual machine, an error notification is sent to the manager of each virtual machine to notify each manager to stop the operation of the virtual machine it manages.

[0072] Combine the following Figure 7 and Figure 4 The AER mechanism of PCIe devices is used as an example to introduce it. Figure 7As shown, when a physical IO device (such as a NIC) detects that an error occurs in VF1 (a target virtualized IO device), the NIC, as a terminal of the bus system 202 (such as a terminal 221B), sends an error message to its upstream port (i.e., port 223B of the switch); after receiving the error message, the switch forwards it to RP222A; RP222A records the error information of the error (including the BDF of VF1) in the AER error register, and at the same time, triggers an AER interrupt through the AER error message notification mechanism, i.e., sends an error interrupt signal to the operating system kernel; the AER error processing driver of the operating system kernel responds to the error interrupt signal, reads the target error information in the AER error register (first register) of RP222A, i.e., the BDF of VF1, and then reads the target error information in the AER error register (second register) of VF1, i.e., the error type information of the target error.

[0073] If the error type of the target error is an isolatable error type such as a transactional error (such as a non-fatal error), the AER error handling driver determines that the target manager is QEMU1 through the BDF and mapping relationship of VF1, and then notifies QEMU1 of the event by calling the error handling function of the VFIO driver, notifying QEMU1 to stop the operation of VF1; QEMU1 sends a virtual machine stop command (vm_stop) to Guest1 by calling its registered error response function to stop the operation of Guest1.

[0074] If the error type of the target error is a non-isolated error type such as a link layer error or a physical layer error (such as a fatal error), the AER error handling driver traverses each virtualized IO device (i.e., VF1, VF2, ...) connected to the upstream port of VF1 (i.e., port 223B of the switch), calls the error handling function of the VFIO driver, and sends an error notification to QEMU1, QEMU2, ... through event notifications to QEMU1, QEMU2, ...; QEMU1 sends a virtual machine stop command to Guest1 by calling its registered error response function to stop the operation of Guest1; QEMU2 sends a virtual machine stop command to Guest2 by calling its registered error response function to stop the operation of Guest1; ...

[0075] In this application example, based on the impact range characteristics of fatal errors and non-fatal errors in the PCIe device link, it is found that non-fatal errors can be isolated between virtual machines, so that the virtual machine to which the erroneous VF belongs will be stopped under non-fatal errors, realizing AER error isolation based on error type, and improving the stability and reliability of the virtualized IO system.

[0076] It should be noted that the above application scenarios or application examples provided in the embodiments of the present application are for ease of understanding, and the embodiments of the present application do not specifically limit the application of the technical solution.

[0077] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0078] Corresponding to the error handling method for IO devices provided in the embodiment of the present application, the embodiment of the present application also provides an error handling device for IO devices, the device comprising: a target error information acquisition module, used to obtain target error information in response to an error interrupt request for a target error, wherein the error interrupt request is triggered when a physical IO device discovers the target error, the physical IO device is virtualized into multiple virtualized IO devices, and the multiple virtualized IO devices are assigned to multiple virtual machines; a target virtual machine stopping operation module, used to stop the operation of a target virtual machine among the multiple virtual machines based on the target error information, wherein the target error information includes associated information associated with the impact scope of the target error, and the target virtual machine is a virtual machine within the impact scope of the target error.

[0079] In one embodiment, the associated information is information about the error type of the target error, and when the error type of the target error is an isolatable error type, the target virtual machine is a virtual machine to which a target virtualized IO device belongs, and the target virtualized IO device is a virtualized IO device where the target error occurs. Exemplarily, the isolatable error type includes a transactional error.

[0080] In one embodiment, the target error information also includes device information of the target virtualized IO device, and the target virtual machine stopping operation module is specifically used to: in response to the error type of the target error being an isolatable error type, determine the target manager corresponding to the device information according to the mapping relationship between the device information of multiple virtualized IO devices and managers of multiple virtual machines; and send an error notification to the target manager, wherein the error notification is used to notify the target manager to stop the operation of the target virtual machine it manages.

[0081] In one embodiment, the association information is information about the error type of the target error, and when the error type of the target error is a non-isolated error type, the target virtual machines are the multiple virtual machines. Exemplarily, the non-isolated error type includes a link layer error or a physical layer error.

[0082] In one embodiment, the target error information also includes device information of a target virtualized IO device, wherein the target virtualized IO device is a virtualized IO device where the target error occurs, and the target virtualized IO device is connected to an upstream port. The target virtual machine operation stopping module is specifically used for: in response to an error type of the target error being a non-isolated error type, traversing each virtualized IO device connected to the upstream port, and sending error notifications to managers of virtual machines corresponding to each virtualized IO device, respectively, wherein the error notifications are used to notify the corresponding manager to stop the operation of the target virtual machine it manages.

[0083] In one embodiment, the associated information is information about the error type of the target error, and the target error information also includes device information of a target virtualized IO device, where the target virtualized IO device is the virtualized IO device where the target error occurs. The target error information acquisition module is specifically used to: read the device information of the target virtualized IO device from a first register, wherein the first register is a register of a bus root port that triggers the error interrupt request; and read the error type information of the target error from a second register, wherein the second register is a register of the target virtualized IO device.

[0084] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0085] Figure 8 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 8 As shown, the electronic device includes: a memory 801 and a processor 802, and the memory 801 stores a computer program that can be run on the processor 802. When the processor 802 executes the computer program, the method in the above embodiment is implemented. The number of the memory 801 and the processor 802 can be one or more. In a specific implementation, the electronic device may also include a communication interface 803 for communicating with external devices and performing data exchange transmission.

[0086] In specific implementation, if the memory 801, the processor 802 and the communication interface 803 are implemented independently, the memory 801, the processor 802 and the communication interface 803 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a PCI bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0087] Optionally, in a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on a chip, the memory 801, the processor 802 and the communication interface 803 can communicate with each other through an internal interface.

[0088] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0089] An embodiment of the present application provides a computer program product, including a computer program, which implements the method provided in the embodiment of the present application when executed by a processor.

[0090] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0091] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0092] It should be understood that the processor may be a CPU, or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0093] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a RAM, which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0094] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0095] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0096] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0097] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0098] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0099] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0100] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0101] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An error handling method for an IO device, comprising: In response to an error interrupt request for a target error, device information of a target virtualized IO device where the target error occurs is read from a first register, and information on the error type of the target error is read from a second register, wherein the error interrupt request is triggered when a physical IO device finds the target error, the physical IO device is virtualized into multiple virtualized IO devices, the multiple virtualized IO devices are allocated to multiple virtual machines, the error type of the target error is an isolatable error type or a non-isolatable error type, the first register is a register of a bus root port that triggers the error interrupt request, and the second register is a register of the target virtualized IO device; Based on the device information and the information of the error type, a target virtual machine among the multiple virtual machines is stopped from running, wherein the target virtual machine is a virtual machine within an impact range associated with the error type of the target error.

2. The error handling method according to claim 1, wherein: In a case where the error type of the target error is the isolatable error type, the target virtual machine is a virtual machine to which the target virtualized IO device belongs.

3. The error handling method according to claim 2, wherein: The isolatable error types include transactional errors.

4. The error handling method according to claim 2, wherein: The target error information also includes device information of the target virtualized IO device. The stopping of the target virtual machine among the multiple virtual machines based on the target error information includes: In response to the error type of the target error being an isolatable error type, determining a target manager corresponding to the device information according to a mapping relationship between device information of multiple virtualized IO devices and managers of multiple virtual machines; An error notification is sent to the target manager, where the error notification is used to notify the target manager to stop the operation of the target virtual machine managed by the target manager.

5. The error handling method according to claim 1, wherein: When the error type of the target error is the non-isolated error type, the target virtual machines are the multiple virtual machines.

6. The method according to claim 5, wherein: The non-isolatable error type includes a link layer error or a physical layer error.

7. The method according to claim 5, wherein: The target error information also includes device information of the target virtualized IO device, and the target virtualized IO device is connected to an uplink port. The stopping of the target virtual machine among the multiple virtual machines based on the target error information includes: In response to the error type of the target error being a non-isolated error type, traverse each virtualized IO device connected to the upstream port, and send error notifications to managers of virtual machines corresponding to each virtualized IO device respectively, wherein the error notifications are used to notify the corresponding manager to stop the operation of the target virtual machine it manages.

8. A host machine, comprising: A physical IO device, wherein the physical IO device is virtualized into a plurality of virtualized IO devices, and the plurality of virtualized IO devices are allocated to a plurality of virtual machines; A processor, the processor carrying the multiple virtual machines, the processor being used to implement the method according to any one of claims 1 to 7 when receiving an erroneous interrupt request.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images