Error handling methods, hosts, devices, media, and products

Abstract

The embodiment of the present application provides an error handling method, a host machine, a device, a medium and a product, which relates to the field of computer technology. The method includes: responding to an error interrupt request for a target error, determining the error source type of the target error, wherein the error interrupt request is triggered by a bus root port when a target error is found, the bus root port is connected to multiple IO devices, and the multiple IO devices are assigned to multiple virtual machines to provide memory access rights for multiple virtual machines, and the error source type is used to indicate the device type of the error source; based on the error source type of the target error, stopping the operation of a target virtual machine among multiple virtual machines, wherein the target virtual machine matches the error source type of the target error. The technical solution of the embodiment of the present application can perform error isolation according to the error source type, thereby improving the stability and reliability of the system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an error handling method, a host machine, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In public clouds, virtual machine downtime rate is a key indicator in the Service Level Agreement (SLA), and error isolation has a significant impact on reducing the downtime rate. However, in the error handling mechanism based on the current operating system kernel, it is impossible to isolate errors between virtual machines, which affects system stability and SLA compliance rate. Summary of the invention

[0003] Embodiments of the present application provide an error handling method, a host machine, an electronic device, a computer-readable storage medium, and a computer program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present application provides an error handling method, comprising: in response to an error interrupt request for a target error, determining an error source type of the target error, wherein the error interrupt request is triggered by a bus root port when the target error is discovered, the bus root port is connected to multiple IO devices, and the multiple IO devices are assigned to multiple virtual machines to provide memory access rights for the multiple virtual machines, and the error source type is used to indicate a device type of the error source; based on the error source type of the target error, stopping the operation of a target virtual machine among the multiple virtual machines, wherein the target virtual machine matches the error source type of the target error.

[0005] In a second aspect, an embodiment of the present application provides a host machine, comprising: multiple IO devices, which are assigned to multiple virtual machines to provide memory access rights for the multiple virtual machines; a processor, which is connected to the IO devices via a bus root port, the processor carries the multiple virtual machines, and is used to implement any method of the embodiments of the present application when an error interrupt request is triggered by the bus root port.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor implements any method of the embodiment of the present application when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any one of the embodiments of the present application is implemented.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any method of the embodiments of the present application when executed by a processor.

[0009] According to the technical solution of the embodiment of the present application, in the memory access scenario, for the target error discovered by the RP, the operating system kernel will determine the error source type of the target error. Since the error source type can indicate the device type of the error source and represent the impact scope of the target error, the corresponding target virtual machine can be matched to stop running according to the error source type of the target error. The target virtual machine may be one or more, thereby realizing on-demand error isolation, thereby improving the stability and reliability of the system.

[0010] Optionally, when the error source type is an IO device error, it is only necessary to stop the operation of the target virtual machine related to the error so that other devices not related to the error will not be affected by the error, thereby achieving error isolation, thereby enabling the device or system to recover from the error as quickly as possible without resetting link components and interfering with other ongoing transactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0012] Figure 1 A schematic diagram of a system architecture of a host machine provided in an embodiment of the present application is shown;

[0013] Figure 2 A schematic diagram showing the working principle of a memory access method in an embodiment of the present application is shown;

[0014] Figure 3 A schematic diagram showing the working principle of another memory access method in an embodiment of the present application

[0015] Figure 4 A flowchart showing an error handling method according to an embodiment of the present application is shown;

[0016] Figure 5 An exemplary schematic diagram showing an event queue;

[0017] Figure 6 A system architecture diagram showing an error handling method according to an embodiment of the present application;

[0018] Figure 7 A timing diagram showing an error handling method according to an embodiment of the present application;

[0019] Figure 8 A system architecture diagram showing an error handling method according to an embodiment of the present application;

[0020] Fig. 9 A timing diagram showing an error handling method according to an embodiment of the present application;

[0021] Fig.10 A block diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0023] To facilitate understanding, first combine Figure 1 , Figure 2 and Figure 3 Introduce the application scenarios of the technical solutions of the embodiments of this application.

[0024] Figure 1 A schematic diagram of the system architecture of a host machine provided by an embodiment of the present application is shown, and the system architecture of the host machine may include a processor 101, a bus system 102, and a memory 103. The processor 101 is, for example, a central processing unit (CPU); the memory 103 is, for example, a random access memory (RAM), which provides a physical address space of the host machine. The number of processors 101 and memory 103 may be one or more, and they are interconnected through the bus system 102 to complete mutual communication.

[0025] When the host machine is started, the processor 101 first reads the startup program to load the host operating system. The host operating system runs in the host kernel space. In the virtualization scenario, the user can start one or more virtual machines 104 in the host user space, including installing the guest operating system on the virtual machine 104. In virtualization technology, Guest can be used to represent a virtual machine.

[0026] The bus system 102 is, for example, a PCI bus system, a peripheral component interconnect express (PCIe) bus system, etc. Taking the PCIe bus system as an example, the system architecture of the bus system 102 generally includes a root complex, a bus, an endpoint, and a switch. The root complex is the root node of the entire bus system 102. On the root complex, there are many root ports (RP), i.e., bus root ports, such as bus root port (RP) 122A and bus root port (RP) 122B. Each RP can be connected to one or more IO devices, and an IO device is a terminal. For example, RP122A connects two IO devices, i.e., IO device 121A and IO device 121B, through a switch, and RP122B connects one IO device, i.e., IO device 121C. In the PCIe bus system, the IO device is a PCIe device.

[0027] The processor 101 is connected to the root complex (i.e., to each RP), and finally to all IO devices. The RP is responsible for managing and controlling the IO devices connected to it. It can pass the IO requests of the IO devices to the processor 101, and send the instructions of the processor 101 to the IO devices. It should be noted that at the hardware level, the root complex can be integrated into the processor 101, such as Figure 2 shown.

[0028] Figure 2 The schematic diagram of the working principle of a memory access method in the embodiment of the present application is shown, wherein the IO devices (including the IO device 121A, the IO device 121B and the IO device 121C) are physical IO devices.

[0029] Direct Memory Access (DMA) is a technology that allows peripheral devices to transfer data without relying on the CPU, greatly improving CPU utilization. However, in a virtualized environment, in the device passthrough scenario, the passthrough device also needs to use DMA technology to access the main memory of the virtual machine to improve I / O performance. At this time, the DMA security and isolation of the passthrough device must be guaranteed, that is, the passthrough device of a virtual machine (VM) cannot access the memory of other VMs through DMA, nor can it directly access the memory of the host, otherwise it will cause extremely serious consequences. In order to solve this problem, the Input / Output Memory Management Unit (IOMMU) technology was introduced. IOMMU technology can also be called Virtualization Technology for Direct I / O (VT-d) technology.

[0030] The main function of IOMMU is to manage the mapping between devices and physical memory, acting as an address translation agent (TA). Figure 2 103 in the memory. In a virtualized scenario, the IOMMU configures an independent mapping page table for each virtual machine to ensure that memory access between multiple virtual machines and between the host machine and the DMA request address is isolated from each other, and that the address of the DMA request is correctly redirected to the physical address space of the virtual machine after address translation. For example, the IOMMU is the System Memory Management Unit (SMMU).

[0031] like Figure 2 As shown, the memory management unit (MMU) of the processor 101 is responsible for managing the access rights and address mapping of the processor 101 to the memory 103. The IO device includes an address translation cache (ATC) for caching the address mapping results recently used by the DMA engine to reduce the address translation delay during each DMA operation; a configuration space (Configuration Space) and a memory mapping (MMIO) register, which reads and writes the configuration information of the IO device through the configuration write / configuration read (CfgWr / CfgRd) operation on the link.

[0032] like Figure 2As shown, in a virtualization scenario, when the virtual machine user state initiates a DMA request, it will use the virtual machine physical address (Guest Physical Address, GPA) as the input / output virtual address (Input / Output Virtual Address, IOVA) of memory 103. The IO device where the virtual machine is located sends a memory read (MRd) or memory write (MWr) request to the RP connected to it through the DMA engine. When the IO device initiates a memory read request, the DMA engine sends an address translation request to the RP connected to it. The root complex will convert the bus address of the IO device into a memory domain address, and the SMMU acts as an address translation agent (Translation Agent, TA). By accessing the address translation and protection table (Address Translation and Protection Table, ATPT), the bus address of the IO device is converted into the host physical address (Host Physical Address, HPA) corresponding to the GPA, and then the memory controller is used to perform memory read and write operations.

[0033] In related technologies, for errors found by RP, whether fatal or non-fatal, the operating system kernel will traverse the bus where the RP is located, call the driver error handling function of each IO device connected to the RP, and stop the operation of the virtual machines to which each IO device connected to the RP belongs, which will affect system stability and SLA compliance. In the public cloud scenario, this means that the virtual machines of other users on the physical machine may be subject to malicious attacks of illegal DMA access.

[0034] Based on the technical solution of the embodiment of the present application, in one example, Figure 2 As shown, the virtual machine kernel to which the IO device 121A belongs uses GPA as IOVA to initiate a DMA request, but the IOVA is an illegal address, that is, the DMA request is an abnormal DMA request. In response to the DMA request, the IO device 121A sends an address translation request to the RP122A connected to it. The SMMU discovers the address translation error by accessing the ATPT and returns the target error to RP122A. RP122A reports the target error it has discovered to the operating system kernel of the processor 101 through the error handling mechanism. The operating system kernel determines that the error source type of the target error is an IO device error, and will find the IO device 121A through the error handling mechanism inside the system, and stop the operation of the virtual machine to which it belongs to achieve error isolation.

[0035] Figure 3A schematic diagram of the working principle of another memory access method in an embodiment of the present application is shown. Among them, the IO devices include physical IO devices (such as physical IO device 121D, physical IO device 121D and physical IO device 121F) and virtualized IO devices, such as VF11, VF12...; VF21, VF22...; VF31, VF32...

[0036] In Single Root Input Output Virtualization (SR-IOV), a physical IO device is divided into multiple virtual functions (VFs). As a virtualized IO device, each VF has an independent bus configuration space and device information (including bus number, device number and function number, referred to as BDF), which can be directly allocated to virtual machines for use, allowing multiple virtual machines to share the same physical IO device.

[0037] In the related technology, if the virtual machine to which VF11 belongs initiates an abnormal DMA request, RP122A reports the error to the operating system kernel based on the error handling mechanism within the system, which eventually causes all virtual machines to which all IO devices connected to RP122A (including VF11, VF12...; VF21, VF22...; VF31, VF32...) belong to stop running, which will affect system stability and SLA compliance rate.

[0038] Based on the technical solution of the embodiment of the present application, in one example, Figure 3 As shown, the virtual machine to which the virtualized IO device (such as VF11) belongs initiates an abnormal DMA request. RP122A reports the target error it finds to the operating system kernel of the processor 101 through the error handling mechanism. The operating system kernel determines that the error source type of the target error is an IO device error, finds VF11 through the error handling mechanism within the system, and stops the operation of the virtual machine to which it belongs, thereby achieving error isolation.

[0039] That is to say, in the embodiment of the present application, for the target error discovered by the RP, the operating system kernel will determine the error source type of the target error, and stop running the corresponding target virtual machine according to the error source type of the target error, thereby achieving error isolation.

[0040] It should be noted that the above application scenarios or application examples provided in the embodiments of the present application are for ease of understanding, and the embodiments of the present application do not specifically limit the application of the technical solution. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0041] The technical solution of the present application is described in detail with specific embodiments below. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0042] Figure 4 FIG. 1 is a flowchart of an error handling method according to an embodiment of the present application, and the error handling method can be applied to an operating system kernel of a processor. Figure 4 As shown, the error handling method may include:

[0043] Step S201: In response to an error interrupt request for a target error, determining an error source type of the target error, wherein the error interrupt request is triggered by the RP when the target error is found, the RP is connected to a plurality of IO devices, the plurality of IO devices are allocated to a plurality of virtual machines to provide memory access rights for the plurality of virtual machines, and the error source type is used to indicate a device type of the error source;

[0044] Step S202: based on the error source type of the target error, stopping the operation of a target virtual machine among the multiple virtual machines, wherein the target virtual machine matches the error source type of the target error.

[0045] Among them, RP is, for example, Figure 1 , Figure 2 and Figure 3 RP122A and RP122B in. The IO device can be a physical IO device, such as Figure 1 and Figure 2 The IO devices 121A, IO devices 121B and IO devices 121C in the embodiment may also be virtualized IO devices, such as Figure 3 VF11, VF12...; VF21, VF22...; VF31, VF32... In terms of functional categories, IO devices can be PCI devices or PCIe devices. PCIe devices include but are not limited to graphics processing units (GPUs), NICs, sound cards, solid state drives (SSDs), etc.

[0046] Exemplarily, multiple IO devices have a one-to-one mapping relationship with multiple virtual machines, that is, one IO device belongs to one virtual machine, and provides the virtual machine with memory access rights to the host machine. In the embodiment of the present application, the memory access right is direct memory access (DMA) permission, that is, the virtual machine can directly access the corresponding IO device to achieve direct memory access.

[0047] In the system error handling mechanism, after the RP finds the target error, it will trigger the error interrupt signal, that is, send the error interrupt signal to the operating system kernel, and the operating system kernel will respond to the error interrupt signal and perform corresponding error handling. Taking the IO device as a PCIe device as an example, the PCIe protocol stipulates the Advanced Error Reporting (AER) mechanism. After the PCIe RP finds the target error, it will trigger the AER interrupt and send the error interrupt signal to the operating system kernel of the processor. The interrupt handling function registered in the AER error handling driver of the operating system kernel will respond to the error interrupt signal and determine the error source of the target error.

[0048] There are two main types of errors reported by the RP. One is that the device information recorded in the AER register of the RP is the BDF of the RP, and the other is that the device information recorded in the AER register of the RP is the BDF of the switch connected to the RP. For example, if the device information recorded in the AER register of the RP is the BDF of the RP, it can be considered that the error reported by the RP is the target error discovered by the RP.

[0049] The error source type is used to indicate the device type of the error source, including but not limited to IO device error or RP error. If it is an IO device error, it means that the error source of the target error may be a certain IO device connected to it, for example, the virtual machine to which a certain IO device belongs initiates an abnormal DMA request, which leads to the target error; if it is an RP error, it means that the error source of the target error may be the RP itself, for example, a fatal error on the RP leads to the target error.

[0050] Different error source types of the target error indicate different impact scopes of the target error, so different error isolation measures can be taken. Therefore, after receiving a target error discovered by a certain RP, the operating system kernel will first determine the error source type of the target error. In one example, the error handling driver of the operating system kernel can read the error information of the target error from the error information register, and then parse the error source type. In other examples, the operating system kernel can obtain the error information of the target error from the event list of the recorded transaction, and parse it to obtain the error source type (which will be described in detail below), and this application is not specifically limited to this.

[0051] The operating system kernel will match the target virtual machine according to the error source type of the target error and stop it. If it is an IO device error, it means that the target error is caused by the target IO device. The operating system kernel will match the virtual machine to which the target IO device belongs as the target virtual machine and stop it. In the end, only one virtual machine will be stopped to achieve error isolation. If it is an RP error, it means that the target error may be a fatal error on the RP. In this case, all virtual machines to which the IO devices connected to the RP belong must be used as target virtual machines and all of these target virtual machines must be stopped.

[0052] According to the technical solution of the embodiment of the present application, for the target error discovered by RP, the operating system kernel will determine the error source type of the target error. Since the error source type can indicate the device type of the error source and represent the impact scope of the target error, the corresponding target virtual machine can be matched to stop running according to the error source type of the target error. The target virtual machine may be one or more, thereby realizing on-demand error isolation, thereby improving the stability and reliability of the system.

[0053] In one embodiment, in step S201, determining the error source type of the target error may include: when the primary error type of the target error is a transactional error, obtaining the secondary error type and device information of the target error from the event queue of the SMMU; in response to the secondary error type of the target error being a memory access address translation error and the device information of the target error being IO device information, determining that the error source type is an IO device error.

[0054] Among them, the first-level error type can be a transactional error, such as a Completer Abort error. Exemplarily, the first-level error type of the target error can be read from the register of the RP that discovered the target error. After discovering the target error, the RP will record the first-level error type of the target error in its register and trigger an error interrupt request. In the AER mechanism, the register of the RP includes the AER register. The address offset of the AER register is specified in the PCIe protocol. The interrupt processing function registered in the AER error handling driver of the operating system kernel can access the register based on the address offset of the register of the RP to read the first-level error type of the target error. If the first-level error type of the target error is a transactional error, the second-level error type and device information of the target error can be further obtained from the event queue of the SMMU.

[0055] The SMMU event queue is a structure used to store error information. When an IO device initiates a memory access request through the SMMU, if the SMMU finds a problem during address translation (such as accessing an illegal address), the SMMU generates an event and puts it into the event queue for further processing by the system. These events usually contain detailed information about the error, such as: secondary error type (such as translation error, permission error, etc.), device identification where the error occurred, and other status information about the error.

[0056] Figure 5 FIG. 2 is an exemplary schematic diagram showing an event queue of an SMMU. Figure 5 As shown, bits 63 to 32 represent an event, recording a stream identifier (ID), which is used to identify a specific data stream or channel. In an embodiment of the present application, the stream ID can record device information, such as BDF. Bits 31 to 0 represent another event, in which bits 31 to 12 record a substream ID, which is used to identify a specific sub-data stream or sub-channel; bit 11 is a reserved field (Reserved 0, RES0); bits 10 to 8 are substream valid bits (SubStream Valid, SSV); bits 7 to 0 are fixed value fields, recording the type code of the event. In an embodiment of the present application, bits 7 to 0 can record the secondary error type.

[0057] Therefore, when an IO device initiates an abnormal memory access request through the SMMU and causes a target error, the SMMU generates a target event and records the secondary error type and device information of the target error. Figure 5 As shown, the secondary error type can be obtained from bits 7 to 0, for example, "0×10" indicates a memory access address translation fault; the device information is the BDF recorded by the stream ID in bits 63 to 32.

[0058] Further, if the secondary error type is a memory access address translation error and the device information is device information of an IO device (IO device information for short), it can be determined that the error source type is an IO device error.

[0059] In the SMMU-based memory access method, the DMA request initiated by the virtual machine is responded to by its corresponding IO device, and the address is translated by the SMMU. The relevant transactions will be recorded in the event list of the SMMU. If the DMA request accesses an illegal address, the error type of the memory access address translation error and the device information of the IO device will be recorded. Therefore, in virtualization technology, the SMMU participates in and records the relevant process information of the DMA operation. The error handling driver of the operating system kernel can obtain the error type and device information of the target error from the event list of the SMMU, and then quickly determine the error source type, thereby simplifying the error handling logic of the operating system kernel and improving the error handling efficiency of the operating system kernel.

[0060] In one embodiment, in step S201, determining the error source type of the target error may include: in response to the first-level error type of the target error being a transactional error, and there being no error event associated with the target error in the event queue of the system memory management unit, determining that the error source type is a bus root port error.

[0061] Similarly, in virtualization technology, SMMU participates in and records relevant process information of DMA operations. Therefore, if an IO device error occurs, the event queue of SMMU will record the error event associated with the IO device error, including the aforementioned secondary error type and device information. Conversely, if there is no error event in the event queue of SMMU, it means that an RP error has occurred. Therefore, when there is no error event in the event queue, the error source type can be determined to be an RP error. Based on this, the error source type can be quickly determined, thereby simplifying the error handling logic of the operating system kernel and improving the error handling efficiency of the operating system kernel.

[0062] In one implementation, when the error source type is an IO device error, the target virtual machine is a virtual machine to which the target IO device belongs, and the target IO device is an IO device that causes the target error in response to an abnormal memory access request.

[0063] If the error source type is an IO device error, it means that the target error is caused by the target IO device responding to the abnormal memory access request of its virtual machine. Therefore, the impact range of the target error involves the virtual machine to which the target IO device belongs. The operating system kernel will match the virtual machine to which the target IO device belongs as the target virtual machine and stop its operation. For example, Figure 3As shown, the virtual machine to which the virtualized IO device (such as VF11) belongs initiates an abnormal DMA request. RP122A reports the target error it finds to the operating system kernel of the processor 101 through the error handling mechanism. The operating system kernel determines that the error source type of the target error is an IO device error, and will find VF11 through the error handling mechanism within the system, and stop the operation of the virtual machine to which it belongs, without having to stop the operation of all virtual machines to which VFs under RP122A belong.

[0064] Based on this, when the error source type is an IO device error, it is only necessary to stop the operation of the virtual machine corresponding to the target IO device, so that other devices not related to the error will not be affected by the error, thereby achieving error isolation, and enabling the device or system to recover from the error as quickly as possible without resetting link components and interfering with other ongoing transactions.

[0065] In one embodiment, in step S202, based on the error source type of the target error, a target virtual machine among multiple virtual machines is stopped from running, including: in response to the error source type being an IO device error, determining a target manager corresponding to the target IO device according to a mapping relationship between multiple IO devices and managers of multiple virtual machines, wherein the target IO device is an IO device that causes a target error in response to an abnormal memory access request; and sending an error notification to the target manager, the error notification being used to notify the target manager to stop the operation of the target virtual machine it manages.

[0066] Exemplarily, if the error source type is an IO device error, the manager corresponding to the BDF can be found from the mapping relationship between multiple IO devices and managers of multiple virtual machines according to the BDF of the target IO device. For example, the manager of each virtual machine usually also has its identifier. Therefore, in a computer device, the mapping relationship between multiple IO devices and managers of multiple virtual machines can be expressed as a mapping relationship between the BDFs of multiple IO devices and the identifiers of managers of multiple virtual machines. The operating system kernel searches for the identifier of the target manager through the BDF of the target IO device, thereby sending an error notification to the target manager to notify the target manager to stop the operation of the target virtual machine.

[0067] The virtual machine manager is, for example, the Quick Emulator (QEMU), which can simulate multiple processor architectures and accelerate the operation of the virtual machine through hardware-assisted virtualization technology.

[0068] Taking the IO device as a PCIe device as an example, in the AER mechanism, the AER error handling driver of the operating system kernel obtains the BDF of the target IO device. In the virtualized input and output (Virtual Function InputOutput, VFIO) driver of the operating system kernel, different IO devices correspond to different error handling functions (pci_error_handlers), and they are mapped to the error response function (vfio_err_notifier_handler) registered by the corresponding virtual machine manager. Therefore, based on this mapping relationship, the AER error handling driver can call the error handling function of the target IO device, send an error notification to the target manager, and notify the target manager to call its registered VFIO error response function to send a virtual machine stop command (vm_stop) to the target virtual machine it manages, so as to stop the operation of the target virtual machine.

[0069] Based on this, when the error source type is an IO device error, only the virtual machine to which the target IO device belongs can be used as the target virtual machine, and the target manager can be found based on the mapping relationship between multiple IO devices and managers of multiple virtual machines. The target manager then stops the operation of the target virtual machine, thereby enabling the management logic in the virtual machine scenario to be used to achieve fast and effective error isolation without adding new management logic for error isolation of the virtual machine.

[0070] Combine the following Figure 6 and Figure 7 This section describes an application example in which the error source type is an IO device error. Figure 6 A system architecture diagram showing an error handling method according to an embodiment of the present application; Figure 7 A timing diagram of the error handling method of an embodiment of the present application is shown. In this application example, the IO devices connected to the bus root port 1 (RP1) are IO device 1 and IO device 2. IO device 1 and IO device 2 can be physical IO devices or virtualized IO devices. The virtual machine to which IO device 1 (hereinafter referred to as dev1) belongs is virtual machine 1 (Guest1), and the virtual machine to which IO device 2 belongs is virtual machine 2 (Guest2). The virtual machine manager is, for example, a Quick Emulator (QEMU), which can simulate multiple processor architectures and accelerate the operation of virtual machines through hardware-assisted virtualization technology. For example, virtual machine manager 1 (QEMU1) is responsible for managing Guest1, and virtual machine manager 2 (QEMU2) is responsible for managing Guest2. The error handling method may include:

[0071] Step S401: The user of Guest 1 initiates a DMA request based on the Guest 1 client, and the DMA request uses the GPA of Guest 1 as the source address IOVA; wherein the source address IOVA is an illegal address;

[0072] Step S402: dev1 sends a transaction layer packet (TLP) requested by MRd to RP1;

[0073] Step S403: RP1 initiates an address translation request to the SMMU, requesting the SMMU to perform IOVA address translation;

[0074] Step S404: Since the source address IOVA is an illegal address, the SMMU accesses the ATPT to query the page table of Guest1, triggers an address translation error event, and puts the event into the event queue. The event also records the secondary error type (memory address translation error) and device information (BDF of dev1).

[0075] Step S405: SMMU responds to RP1 that the translation error occurred;

[0076] Step S406: RP1 finds the translation error (i.e., target error) based on the response of SMMU, i.e., dev1 is the target IO device that causes the target error. Since the target error is a response failure error to dev1's transactional request, it is a transactional error. Therefore, RP1 will record the first-level error type as a transactional error (e.g., CompleterAbort) in RP1's AER register, and will record the BDF of the device that found the target error, i.e., RP1's BDF.

[0077] Step S407: RP1 returns a request response TLP to dev1, and the request response result is Completer Abort, so that the user knows that a Completer Abort error has occurred;

[0078] Step S408: RP1 triggers an AER interrupt, i.e., sends an error interrupt request to the operating kernel, notifying the AER error processing driver of the Host operating system kernel that an error (i.e., a target error) has occurred;

[0079] Step S409: The AER error processing driver responds to the error interrupt signal, reads the first-level error type as transaction error Completer Abort from the AER register of RP1, and records the BDF of the device that found the target error, that is, the BDF of RP1. Then, the error event is obtained from the event queue of the SMMU, and the error event records the second-level error type and device information of the target error;

[0080] Step S410: SMMU returns an error event to the AER error handling driver;

[0081] Step S411: the AER error processing driver parses the error event to find that the secondary error type of the target error is a memory address translation error, and the device information is the BDF of dev1 (target IO device), thereby determining that the error source type is an IO device error (Endpoint fault);

[0082] Step S412: the AER error handling driver calls the error handling function of dev1 in the VFIO driver;

[0083] Step S413: the error handling function sends an error notification to QEMU1 (target manager) via an event notification (eventfd);

[0084] Step S414: QEMU1 calls its registered VFIO error response function to send a virtual machine stop command (such as vm_stop) to the target virtual machine it manages (ie, Guest1) to stop the operation of Guest1.

[0085] In one embodiment, when the error source type is an RP error, the target virtual machine is each IO device connected to the RP that discovers the target error, that is, the multiple virtual machines described in step S201. That is, if the error source type is an RP error, the impact range of the target error will involve each IO device connected to the RP, so it is necessary to stop the operation of each IO device connected to the RP that discovers the target error, so as to avoid other service errors in the system and affect the system reliability.

[0086] In one embodiment, in step S202, based on the error source type of the target error, a target virtual machine among multiple virtual machines is stopped, including: in response to the error source type being an RP error, traversing each IO device connected to the RP, and sending error notifications to the managers of the virtual machines corresponding to each IO device, respectively, wherein the error notifications are used to notify the corresponding manager to stop the operation of the target virtual machine it manages.

[0087] When the error source type is an RP error, the operating system kernel needs to traverse each IO device connected to the RP that finds the target error, and send error notifications to the managers of these IO devices respectively, taking the virtual machines to which these IO devices belong as target virtual machines and stopping the operation of these target virtual machines.

[0088] The operating system kernel does not know the information of the IO devices, so it can traverse each IO device connected to it based on the device information of the RP, and then send error notifications to the corresponding virtual machine managers to stop the operation of the virtual machine. In this way, without changing the error handling mechanism of the operating system kernel, the operation of the virtual machine affected by the RP error is stopped to avoid other service errors in the system and affect the system reliability.

[0089] Combine the following Figure 8 and Fig. 9 This section describes an application example when the error source type is an RP error. Figure 8 A system architecture diagram showing an error handling method according to an embodiment of the present application; Fig. 9 A timing diagram of an error handling method according to an embodiment of the present application is shown. The error handling method may include:

[0090] Step S501: During the processing of the DMA operation, RP1 finds a target error, RP1 will record the first-level error type as a transactional error, such as a malformat TLP, in the AER register of RP1, and will record the BDF of the device that found the target error, i.e., the BDF of RP1, and RP1 triggers an AER interrupt, i.e., sends an error interrupt request to the operating kernel, notifying the AER error handling driver of the host operating system kernel that an error (i.e., a target error) has occurred;

[0091] Step S502: The AER error processing driver responds to the error interrupt signal, reads the primary error type as a transaction error (such as Malformat TLP) from the AER register of RP1, and further obtains an error event from the event queue of the SMMU. The error event records the secondary error type and device information of the target error.

[0092] Step S503: SMMU returns an error event to the AER error handling driver;

[0093] Step S504: There is no error event in the SMMU, so the error source type is determined to be an RP error;

[0094] Step S505: the AER error handling driver traverses each IO device connected to RP1, namely, IO device 1 (dev1) and IO device 2 (dev2), and calls the error handling function of dev1 and the error handling function of dev2 in the VFIO driver;

[0095] Step S506: the error handling function of dev1 sends an error notification to QEMU1 through event notification; in parallel, the error handling function of dev2 sends an error notification to QEMU2 through event notification;

[0096] Step S507: QEMU1 calls its registered VFIO error response function to send a virtual machine stop command to Guest1 it manages, so as to stop the operation of Guest1; in parallel, QEMU2 calls its registered VFIO error response function to send a virtual machine stop command to Guest2 it manages, so as to stop the operation of Guest2.

[0097] Corresponding to the error handling method provided in the embodiment of the present application, the embodiment of the present application also provides an error handling device, which includes: an error source type determination module, used to determine the error source type of the target error in response to an error interrupt request for the target error, wherein the error interrupt request is triggered by the bus root port when the target error is found, the bus root port is connected to multiple IO devices, and the multiple IO devices are assigned to multiple virtual machines to provide memory access rights for the multiple virtual machines, and the error source type is used to indicate the device type of the error source; a target virtual machine stopping operation module, used to stop the operation of a target virtual machine among the multiple virtual machines based on the error source type of the target error, wherein the target virtual machine matches the error source type of the target error.

[0098] In one embodiment, the error source type determination module is specifically used to: when the primary error type of the target error is a transactional error, obtain the secondary error type and device information of the target error from the event queue of the system memory management unit; in response to the secondary error type of the target error being a memory access address translation error and the device information of the target error being IO device information, determine that the error source type is an IO device error.

[0099] In one implementation, when the error source type is an IO device error, the target virtual machine is a virtual machine to which a target IO device belongs, and the target IO device is an IO device that causes the target error in response to an abnormal memory access request.

[0100] In one embodiment, the target virtual machine stopping operation module is specifically used for: in response to the error source type being an IO device error, determining the target manager corresponding to the target IO device according to the mapping relationship between the multiple IO devices and the managers of the multiple virtual machines, wherein the target IO device is an IO device that causes the target error in response to an abnormal memory access request; and sending an error notification to the target manager, wherein the error notification is used to notify the target manager to stop the operation of the target virtual machine it manages.

[0101] In one embodiment, the error source type determination module is specifically used to: in response to the first-level error type of the target error being a transactional error, and there being no error event associated with the target error in the event queue of the system memory management unit, determine that the error source type is a bus root port error.

[0102] In one implementation, when the error source type is the bus root port error, the target virtual machine is the multiple virtual machines.

[0103] In one embodiment, the target virtual machine stopping operation module is specifically used to: in response to the error source type being a bus root port error, traverse each IO device connected to the bus root port, and send the error notification to the manager of the virtual machine corresponding to each IO device respectively, and the error notification is used to notify the corresponding manager to stop the operation of the target virtual machine it manages.

[0104] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0105] Fig.10 is a block diagram of an electronic device used to implement an embodiment of the present application. Fig.10 As shown, the electronic device includes: a memory 1001 and a processor 1002, and the memory 1001 stores a computer program that can be run on the processor 1002. When the processor 1002 executes the computer program, the method in the above embodiment is implemented. The number of the memory 1001 and the processor 1002 can be one or more. In a specific implementation, the electronic device may also include a communication interface 1003 for communicating with an external device and performing data exchange transmission.

[0106] In specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are implemented independently, the memory 1001, the processor 1002 and the communication interface 1003 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a PCI bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0107] Optionally, in a specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are integrated on a chip, the memory 1001, the processor 1002 and the communication interface 1003 can communicate with each other through an internal interface.

[0108] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0109] An embodiment of the present application provides a computer program product, including a computer program, which implements the method provided in the embodiment of the present application when executed by a processor.

[0110] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0111] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0112] It should be understood that the processor may be a CPU, or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0113] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a RAM, which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0114] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0115] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0116] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0117] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0118] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0119] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0120] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0121] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An error handling method, comprising: In response to an error interrupt request for a target error, determining an error source type of the target error according to a primary error type and a secondary error type of the target error, wherein the error interrupt request is triggered by a bus root port when the target error is found, the bus root port is connected to a plurality of IO devices, the plurality of IO devices are allocated to a plurality of virtual machines to provide memory access rights for the plurality of virtual machines, and the error source type is used to indicate a device type of the error source, including an IO device error or a bus root port error; Based on the error source type of the target error, a target virtual machine among the multiple virtual machines is stopped from running, wherein the target virtual machine matches the error source type of the target error.

2. The error handling method according to claim 1, wherein: The step of determining the error source type of the target error according to the primary error type and the secondary error type of the target error includes: When the primary error type of the target error is a transactional error, acquiring the secondary error type and device information of the target error from an event queue of a system memory management unit; In response to the secondary error type of the target error being a memory access address translation error and the device information of the target error being IO device information, it is determined that the error source type is an IO device error.

3. The error handling method according to claim 2, wherein: In the case where the error source type is an IO device error, the target virtual machine is a virtual machine to which a target IO device belongs, and the target IO device is an IO device that causes the target error in response to an abnormal memory access request.

4. The error handling method according to any one of claims 1 to 3, wherein: The step of stopping the operation of a target virtual machine among the multiple virtual machines based on the error source type of the target error includes: In response to the error source type being an IO device error, determining a target manager corresponding to a target IO device according to a mapping relationship between the multiple IO devices and managers of the multiple virtual machines, wherein the target IO device is an IO device that causes the target error in response to an abnormal memory access request; An error notification is sent to the target manager, where the error notification is used to notify the target manager to stop the operation of the target virtual machine managed by the target manager.

5. The error handling method according to claim 1, wherein: Determining the error source type of the target error based on the primary error type and the secondary error type of the target error includes: in response to the primary error type of the target error being a transactional error, and there being no error event associated with the target error in an event queue of a system memory management unit, determining that the error source type is a bus root port error.

6. The error handling method according to claim 5, wherein: When the error source type is a bus root port error, the target virtual machines are the multiple virtual machines.

7. The error handling method according to claim 1, 5 or 6, wherein: The step of stopping the operation of a target virtual machine among the multiple virtual machines based on the error source type of the target error includes: In response to the error source type being a bus root port error, traverse each IO device connected to the bus root port, and send error notifications to managers of virtual machines corresponding to each IO device respectively, wherein the error notifications are used to notify the corresponding manager to stop the operation of the target virtual machine it manages.

8. A host machine, comprising: A plurality of IO devices, wherein the plurality of IO devices are allocated to a plurality of virtual machines to provide memory access permissions for the plurality of virtual machines; A processor, wherein the processor is connected to the IO device via a bus root port, the processor carries the multiple virtual machines, and is used to implement the method described in any one of claims 1 to 6 when an error interrupt request is triggered by the bus root port.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images