Hypervisor-based client operating system deadlock debugging method and electronic device

Abstract

The present disclosure relates to a client operating system deadlock debugging method and electronic device based on a hypervisor, wherein the client operating system runs tasks on a multi-core processor, and the method comprises: detecting whether the client operating system is deadlocked; wherein the deadlock of the client operating system refers to the deadlock of a processor in the multi-core processor; if the client operating system is deadlocked, a virtual fast interrupt is injected into the deadlocked client operating system through the hypervisor, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process. The above scheme is used to solve the technical problem in the prior art that the client operating system running on the hypervisor cannot detect deadlock, and provides a reliable information collection and analysis method for the previously difficult to analyze dead loop fault of shutting down interrupts, which can greatly shorten the fault resolution time and improve efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of fault handling, and in particular to a hypervisor-based client operating system deadlock debugging method and electronic device. Background Art

[0002] A hypervisor, also known as a virtual machine monitor (VM), is an intermediate software layer between the underlying physical hardware and the operating system. The hypervisor enables the running of one or more operating systems on a set of underlying physical hardware. The operating system running on the hypervisor is called a guest OS.

[0003] An operating system (OS) is a set of interrelated system software programs that manage and control computer operations, utilize and run hardware and software resources, and provide common services to organize user interactions. It is also the core and cornerstone of a computer system. There are many types of operating systems, and Linux is one such guest OS.

[0004] Operating systems are inherently interrupt-driven. Under unusual circumstances, such as software bugs, certain kernel processes or threads may enter an interrupt-disabled infinite loop or run for extended periods of time with interrupts disabled. Disabling interrupts prohibits the processor from responding to interrupt requests from the interrupt source. Current operating systems running on multi-core processors typically rely on interrupts to synchronize shared resources between different CPUs. Since a CPU no longer responds to interrupts when interrupts are disabled, processes running on other CPUs in the system cannot access shared resources and cannot be scheduled, causing functional anomalies. Because the CPU does not respond to external events when interrupts are disabled, the system appears to be stuck on that CPU, causing processes on other CPUs to deadlock, leading to malfunctioning of the entire system. However, the operating system itself has no way of tracing the point of failure.

[0005] On some other architectures, such as x86, non-maskable interrupts (nmi) are provided to solve similar problems. However, similar non-maskable interrupts (nmi) are not currently supported on the ARM platform.

[0006] Some operating systems, such as the Linux kernel, provide the lockdep kernel deadlock detection method. However, enabling this detection method significantly degrades system performance and is not applicable in officially released versions. Other operating systems, such as some RTOSs, do not provide deadlock detection methods. Summary of the Invention

[0007] The present invention aims to provide a hypervisor-based deadlock debugging method and electronic device for a client operating system, so as to solve the technical problem in the prior art that a client operating system running on a hypervisor cannot detect deadlock.

[0008] To achieve the above objectives, the present disclosure provides, in a first aspect, a hypervisor-based deadlock debugging method for a client operating system, wherein the client operating system runs a task on a multi-core processor, the method comprising:

[0009] Detecting whether a deadlock occurs in the client operating system; wherein the deadlock in the client operating system refers to a deadlock in a processor in the multi-core processor;

[0010] If the client operating system is deadlocked, a virtual fast interrupt is injected into the deadlocked client operating system through the hypervisor, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process.

[0011] Optionally, detecting whether a deadlock occurs in the client operating system includes:

[0012] The first processor in the client operating system sends an inter-processor interrupt (IPI) to the second processor in the client operating system to synchronize shared resources; wherein the task running on the first processor shares resources with the second processor;

[0013] If the second processor does not respond to the IPI interrupt sent by the first processor, it is determined that the second processor is deadlocked, and further determined that the client operating system is deadlocked.

[0014] Optionally, injecting a virtual fast interrupt into the deadlocked client operating system through the hypervisor, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process, including:

[0015] The first processor takes a deadlock detection command as a first parameter and a CPU ID of the second processor as a second parameter, executes a trap instruction and traps into the hypervisor for execution;

[0016] The hypervisor performs a virtual fast interrupt injection operation on the deadlocked virtual processor based on the deadlock detection command and the CPU ID; the hypervisor sets a virtual interrupt suspension on the deadlocked virtual processor, so that when execution is switched from the hypervisor to the deadlocked virtual processor, the virtual interrupt suspension is detected, and the hypervisor jumps to the fast interrupt exception entry of the virtual processor to execute a registered fast interrupt handler.

[0017] Optionally, the hypervisor performs a virtual fast interrupt injection operation on the deadlocked virtual processor based on the deadlock detection command and the CPU ID, including:

[0018] The hypervisor searches for the corresponding physical processor through the deadlocked CPU ID. If the physical processor corresponding to the CPU ID is the currently running physical processor, the hypervisor directly performs the fast interrupt injection operation on the deadlocked virtual processor.

[0019] Optionally, the hypervisor performs a virtual fast interrupt injection operation on the deadlocked virtual processor based on the deadlock detection command and the CPU ID, including:

[0020] The hypervisor finds the corresponding physical processor through the deadlocked CPU ID. If the physical processor corresponding to the CPU ID is not the currently running physical processor, the hypervisor sends an IPI interrupt to the physical processor corresponding to the CPU ID, so that after receiving the IPI interrupt, the physical processor corresponding to the CPU ID performs the fast interrupt injection operation on the deadlocked virtual processor.

[0021] Optionally, the method further includes:

[0022] When executing the fast interrupt handler, a stack backtrace is performed using the program counter value and the connection register value of the current virtual processor, and an execution call chain of the current virtual processor is stored, so that the hypervisor can read the execution call chain and analyze the deadlock cause of the current virtual processor.

[0023] A second aspect of the present disclosure provides a hypervisor-based client operating system deadlock debugging device, comprising:

[0024] A detection module, configured to detect whether a deadlock occurs in a client operating system; wherein the deadlock occurs in the client operating system refers to a deadlock occurs in a processor in a multi-core processor;

[0025] The interrupt injection module is used to inject a virtual fast interrupt into the deadlocked client operating system through the hypervisor when the deadlocked client operating system occurs, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process.

[0026] A third aspect of the present disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when the program is executed by a processor.

[0027] A fourth aspect of the present disclosure provides an electronic device, including:

[0028] a memory having a computer program stored thereon;

[0029] A processor is used to execute the computer program in the memory to implement the steps of the method of the first aspect.

[0030] In the solution of the disclosed embodiments, when a deadlock occurs in a guest operating system (hereinafter referred to as the guest OS), the hypervisor running on EL2 (exception level 2) utilizes interrupt virtualization to inject a virtual fast interrupt (hereinafter referred to as virtualFIQ) into the guest OS running on EL1 (exception level 1). Because IRQ (interrupt) and FIQ (fast interrupt) in the ARM architecture use different exception vector table entries, the interrupt-disabled virtual processor (hereinafter referred to as virtual CPU) in the guest OS can also respond to the fast interrupt and enter the FIQ exception handling process. Information collected within the exception handling process is used to analyze the cause of the interrupt-disabled deadlock, thereby resolving the existing technical problem of guest OSes running on hypervisors being unable to detect deadlocks. This provides a reliable information collection and analysis method for previously difficult-to-analyze interrupt-disabled deadloop faults, significantly reducing troubleshooting time and improving efficiency.

[0031] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0033] Figure 1 This is a flowchart of a method for debugging a deadlock of a client operating system based on a hypervisor according to an exemplary embodiment;

[0034] Figure 2 is a block diagram illustrating a hypervisor-based deadlock debugging method for a client operating system according to an exemplary embodiment;

[0035] Figure 3 is another flow chart of a hypervisor-based deadlock debugging method for a client operating system according to an exemplary embodiment;

[0036] Figure 4 The figure is a block diagram showing a deadlock debugging device according to an exemplary embodiment. DETAILED DESCRIPTION

[0037] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0038] The embodiment of the present disclosure provides a deadlock debugging method for a client operating system based on a hypervisor, wherein the client operating system runs tasks on a multi-core processor, such as Figure 1 As shown, the method includes the following steps.

[0039] Step 101 , detecting whether a deadlock occurs in the client operating system; wherein the deadlock occurs in the client operating system refers to a deadlock occurs in a processor in the multi-core processor.

[0040] Step 102: If the client operating system is deadlocked, a virtual fast interrupt is injected into the deadlocked client operating system through the hypervisor, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process.

[0041] The above solution provides a general mechanism for detecting operating system deadlocks in guest operating systems. When a deadlock occurs in a guest operating system, the hypervisor running on EL2 leverages the interrupt virtualization feature to inject a virtual FIQ into the guest operating system running on EL1. This triggers the CPU, which has disabled interrupts, to respond and initiate the FIQ exception handling process. The cause of the deadlock can then be determined within the exception handling process.

[0042] Next, combine Figure 2 and Figure 3 The method in the embodiment of the invention is described, which includes the following steps.

[0043] Step 301: The ARM platform is powered on and starts up, and enters el2 to run the hypervisor.

[0044] Step 302: The hypervisor loads the guest OS and enters el1 to run.

[0045] In the disclosed embodiment, the guest OS runs tasks on multiple cores; the guest OS installs a fiq interrupt handler handle_fiq at the fiq interrupt entry of the exception vector table; because when the CPU generates a fiq exception, it jumps to the fiq entry of the exception vector table for execution, so the handler handle_fiq can be installed in advance at the fiq interrupt entry;

[0046] Step 303: A shutdown deadloop error occurs in the second processor (hereinafter referred to as vcpu1) in the guest OS.

[0047] Step 304: A task of the first processor (hereinafter referred to as vcpu0) in the guest OS shares resources with vcpu1. vcpu0 sends an IPI interrupt to cpu1 to synchronize the shared resources.

[0048] Step 305: If vcpu1 in the guest OS does not respond to the IPI interrupt sent by vcpu0, vcpu0 uses the deadlock detection command as the first parameter and the CPU ID of vcpu1 as the second parameter to execute a hyp call instruction to trap into the hypervisor for execution.

[0049] In step 306, the hypervisor processes the deadlock detection command. Since the CPU in the guest OS is only a virtual CPU, not an actual physical CPU, the interrupt of the physical CPU is managed in the hypervisor and is not disabled. Therefore, the hypervisor finds the corresponding physical CPU through the ID of the deadlocked virtual CPU and determines whether vcpu1 is running on the currently running physical CPU. If so, step 307 is executed; if not, step 308 is executed.

[0050] Step 307: If it is the currently running physical CPU, the virtualfiq injection operation is directly performed on the deadlocked virtual CPU;

[0051] When executing the virtual fiq injection operation, the hypervisor sets the CPU to have a virtual interrupt pending virtualfiq pending. When switching from the hypervisor to the deadlocked virtual cpu for execution, the ARM virtualization mechanism detects the existence of virtual fiq pending and jumps to the virtual cpu's fiq exception entry for execution;

[0052] Step 308: If the physical CPU corresponding to the deadlocked virtual CPU ID is not the currently running physical CPU, an IPI interrupt is sent to the physical CPU. After the physical CPU corresponding to the deadlocked virtual CPU receives the IPI interrupt, the virtual fiq injection operation in step 8 is performed on the deadlocked virtual CPU.

[0053] Step 309: After virtual fiq is injected, when returning to the guest OS, a fiq exception is triggered, and the guest OS is jumped to the fiq interrupt entry to execute the registered handler handle_fiq;

[0054] In handle_fiq, the stack trace is performed using the current virtual CPU's pc and lr register values, writing the execution call chain to the black box for analysis by the hypervisor. pc refers to the program counter, and lr refers to the link register.

[0055] In the solution of the disclosed embodiment, when a guest OS running on an ARM64 platform experiences a shutdown deadloop, the hypervisor injects virtual FIQ (Fiq) to obtain detailed information about the guest OS failure point. This involves registering an FIQ handler at the guest OS FIQ exception entry point. The normally operating virtual CPU notifies the hypervisor of the deadlocked virtual CPU sequence number (e.g., CPU ID). The hypervisor then injects virtual FIQ into the deadlocked virtual CPU. Finally, the deadlocked virtual CPU executes the FIQ exception handler, providing detailed information for deadlock cause analysis.

[0056] The technical solution proposed in this disclosure not only detects which virtual CPU in the guest OS is experiencing a shutdown deadloop, but also provides detailed information such as the register values ​​and execution path of the faulty virtual CPU for fault analysis. Furthermore, it is universally applicable to the ARM64 platform, making it easy to integrate with various operating systems. It also provides a reliable information collection and analysis method for previously difficult-to-analyze shutdown deadloop faults, significantly reducing troubleshooting time and improving efficiency.

[0057] Based on the same inventive concept, the embodiment of the present disclosure further provides a hypervisor-based client operating system deadlock debugging device 400, such as Figure 4As shown, it includes: a detection module 401, which is used to detect whether a deadlock occurs in the client operating system; wherein, the deadlock of the client operating system refers to the deadlock of the processor in the multi-core processor; an interrupt injection module 402, which is used to inject a virtual fast interrupt into the deadlocked client operating system through the hypervisor when the deadlock occurs in the client operating system, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process.

[0058] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0059] In an exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the aforementioned hypervisor-based client operating system deadlock debugging method. For example, the computer-readable storage medium may be the aforementioned memory including the program instructions. The program instructions may be executed by a processor of an electronic device to implement the aforementioned hypervisor-based client operating system deadlock debugging method.

[0060] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned hypervisor-based guest operating system deadlock debugging method when executed by the programmable device.

[0061] In another exemplary embodiment, an electronic device is provided, comprising a memory storing a computer program; and a processor configured to execute the computer program in the memory to implement the above-mentioned hypervisor-based client operating system deadlock debugging method.

[0062] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.

[0063] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0064] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A hypervisor-based deadlock debugging method for a client operating system, characterized in that: The client operating system runs a task on a multi-core processor, and the method includes: Detecting whether a deadlock occurs in the client operating system; wherein the deadlock in the client operating system refers to a deadlock in a processor in the multi-core processor; If the client operating system is deadlocked, injecting a virtual fast interrupt into the deadlocked client operating system through the hypervisor, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process; Detecting whether a deadlock occurs in the client operating system includes: The first processor in the client operating system sends an inter-processor interrupt (IPI) to the second processor in the client operating system to synchronize shared resources; wherein the task running on the first processor shares resources with the second processor; If the second processor does not respond to the IPI interrupt sent by the first processor, it is determined that the second processor is deadlocked, and further determined that the client operating system is deadlocked.

2. The method according to claim 1, wherein Injecting a virtual fast interrupt into the deadlocked client operating system through the hypervisor, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter a fast interrupt exception handling process, including: The first processor takes a deadlock detection command as a first parameter and a CPU ID of the second processor as a second parameter, executes a trap instruction and traps into the hypervisor for execution; The hypervisor performs a virtual fast interrupt injection operation on the deadlocked virtual processor based on the deadlock detection command and the CPU ID; the hypervisor sets a virtual interrupt suspension on the deadlocked virtual processor, so that when execution is switched from the hypervisor to the deadlocked virtual processor, the virtual interrupt suspension is detected, and the hypervisor jumps to the fast interrupt exception entry of the virtual processor to execute a registered fast interrupt handler.

3. The method according to claim 2, wherein The hypervisor performs a virtual fast interrupt injection operation on the deadlocked virtual processor based on the deadlock detection command and the CPU ID, including: The hypervisor searches for the corresponding physical processor through the deadlocked CPU ID. If the physical processor corresponding to the CPU ID is the currently running physical processor, the hypervisor directly performs the fast interrupt injection operation on the deadlocked virtual processor.

4. The method according to claim 2, wherein The hypervisor performs a virtual fast interrupt injection operation on the deadlocked virtual processor based on the deadlock detection command and the CPU ID, including: The hypervisor finds the corresponding physical processor by using the deadlocked CPU ID. If the physical processor corresponding to the CPU ID is not the currently running physical processor, the hypervisor sends an IPI interrupt to the physical processor corresponding to the CPU ID, so that after receiving the IPI interrupt, the physical processor corresponding to the CPU ID performs the fast interrupt injection operation on the deadlocked virtual processor.

5. The method according to claim 3 or 4, wherein: The method further comprises: When executing the fast interrupt handler, a stack backtrace is performed using the program counter value and the connection register value of the current virtual processor, and an execution call chain of the current virtual processor is stored, so that the hypervisor can read the execution call chain and analyze the deadlock cause of the current virtual processor.

6. A client operating system deadlock debugging device based on a hypervisor, characterized in that: include: A detection module is configured to detect whether a deadlock occurs in a client operating system; wherein the deadlock in the client operating system refers to a deadlock in a processor in a multi-core processor; detecting whether the deadlock occurs in the client operating system includes: The first processor in the client operating system sends an inter-processor interrupt (IPI) to the second processor in the client operating system to synchronize shared resources; wherein the task running on the first processor shares resources with the second processor; If the second processor does not respond to the IPI interrupt sent by the first processor, it is determined that the second processor is deadlocked, and further determined that the client operating system is deadlocked; The interrupt injection module is used to inject a virtual fast interrupt into the deadlocked client operating system through the hypervisor when the deadlocked client operating system occurs, so that the deadlocked client operating system can respond to the virtual fast interrupt and enter the fast interrupt exception handling process.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 5.

Images