PreemptRT-based KVM real-time optimization method

Abstract

The invention discloses a PreemptRT-based KVM (Keyboard Video Mouse) real-time optimization method, which comprises the following steps of: S1, virtualization layer fine-grained preemption optimization: reconstructing a KVM virtualization layer lock mechanism and a processing flow, realizing fine-grained preemption through lock replacement, compression of a critical zone and insertion of a preemption point, and shortening a preemption window; s2, interrupting a real-time forwarding architecture: optimizing an interrupt forwarding path through double-layer interrupt priority mapping, direct injection architecture and interrupt isolation optimization; step S3, optimizing vCPU scheduling real-time performance: improving vCPU scheduling certainty by configuring a real-time scheduling strategy, strong affinity binding and a high-precision awakening means; and S4, memory virtualization low delay: optimizing memory virtualization delay by combining large page pre-allocation and locking, memory lock and TLB refresh optimization and DMA lock granularity optimization.

Description

Technical Field

[0001] This invention relates to the field of operating system technology, and in particular to a KVM real-time optimization method based on Preempt_RT. Background Technology

[0002] With the technological upgrades in key areas such as industrial control, real-time data acquisition, and industrial robot control, virtualization technology, due to its advantages of hardware resource isolation and efficient reuse, has gradually become a core supporting technology for these scenarios. KVM (Kernel Virtual Machine), as a mainstream open-source virtualization solution, is widely used in the construction of related systems. These key scenarios place dual core demands on virtualization technology: "hardware resource isolation" and "deterministic real-time response," requiring virtual machines to have microsecond-level latency response capabilities and extremely low latency jitter.

[0003] The Preempt_RT kernel, through its core features such as full-path preemption, interrupt threading, and priority inheritance, provides underlying technical support for improving system real-time performance and can theoretically serve as the foundation for KVM real-time enhancement. However, traditional KVM designs have not been adapted or optimized for the Preempt_RT kernel. The inherent defects in its own architecture cannot effectively coordinate with the real-time capabilities of the Preempt_RT kernel, making it difficult for existing KVM based on the standard Linux kernel to meet the real-time requirements of critical domains. Therefore, a targeted optimization scheme is urgently needed to achieve deep integration of the two and overcome the application bottleneck of virtualization technology in real-time scenarios.

[0004] In existing technologies, both traditional KVM solutions and preliminary solutions that simply combine KVM with the Preempt_RT kernel have many shortcomings. The core drawbacks are concentrated in the following four aspects: (1) Insufficient preemption capability of virtualization layer: KVM core critical sections (such as VM operation control and memory mapping management) rely on standard spinlock locks. Even in the Preempt_RT kernel environment, such locks will still disable preemption, causing high-priority tasks to be blocked for a long time. The preemption window of the VM-Exit / VM-Entry processing path can reach millisecond level, causing serious scheduling delays.

[0005] (2) Poor real-time performance of interrupt forwarding: Traditional KVM uses the kvm_irqfd interrupt injection mechanism. The peripheral interrupt forwarding path is not threaded and the interrupt priority is not isolated, resulting in a serious priority loss problem in interrupt response. In industrial scenarios, the peripheral interrupt forwarding delay jitter often exceeds 100μs, which cannot meet the low latency requirements of real-time Ethernet, PCIe devices, etc.

[0006] (3) Insufficient determinism in vCPU scheduling: As the core scheduling unit of KVM, the qemu-kvm process corresponding to vCPU adopts the same scheduling strategy as the host ordinary process, which leads to CPU resource contention. At the same time, frequent vCPU migration will cause cache invalidation, and the delay uncertainty of the blocking / wake-up process is high, which seriously affects the running stability of real-time tasks in the virtual machine.

[0007] (4) Large fluctuations in memory virtualization latency: When KVM implements memory virtualization based on EPT / NPT, there are problems such as large page table switching overhead, inaccurate TLB refresh, and intense memory lock contention. In addition, the fragmentation of memory allocation and possible memory swapping result in virtual machine memory access latency fluctuations of up to 10-200μs, which cannot meet the needs of real-time data storage, DMA transmission and other scenarios. Summary of the Invention

[0008] To address the aforementioned problems, this invention provides a KVM real-time optimization method based on Preempt_RT, aiming to solve the core pain points of existing KVM technologies in real-time scenarios. Specifically, the technical problems to be solved include: 1. How to reconstruct the KVM virtualization core layer architecture to adapt to the preemptive features of the Preempt_RT kernel, shorten the preemption window of the virtualization layer (such as VM-Exit processing), solve the priority inversion problem, and improve the preemptibility and response speed of the virtualization layer.

[0009] 2. How to design a real-time interrupt virtualization forwarding mechanism to achieve priority linkage between physical interrupts and virtual machine interrupts, shorten the interrupt forwarding path, reduce interrupt forwarding latency and jitter, and meet the low-latency interrupt response requirements of industrial peripherals.

[0010] 3. How to optimize vCPU scheduling strategy to achieve precise binding between vCPU and physical CPU resources, improve the determinism of vCPU scheduling, and avoid the impact of cache invalidation and resource contention on real-time performance.

[0011] 4. How to optimize the memory virtualization process, reduce the latency caused by page table switching, TLB refresh and memory lock contention, achieve deterministic control of memory access latency, and meet the stability requirements of memory read and write in real-time scenarios.

[0012] This invention is implemented as follows: A KVM real-time optimization method based on Preempt_RT is proposed, which performs targeted adaptation and collaborative optimization in four aspects of KVM: virtualization layer, interrupt forwarding, vCPU scheduling, and memory virtualization. It constructs a three-level real-time enhancement architecture with underlying kernel support, intermediate virtualization enhancement, and upper-layer scenario adaptation through four means: fine-grained preemption optimization of virtualization layer, interrupt real-time forwarding architecture, vCPU scheduling real-time optimization, and memory virtualization low latency. Among them, fine-grained preemption optimization of virtualization layer is implemented through step S1, interrupt real-time forwarding architecture is implemented through step S2, vCPU scheduling real-time optimization is implemented through step S3, and memory virtualization low latency is implemented through step S4. Step S1, fine-grained preemption optimization of the virtualization layer: reconstruct the locking mechanism and processing flow of the KVM virtualization layer, and achieve fine-grained preemption by lock replacement, compression of critical sections and insertion of preemption points, thereby shortening the preemption window; Step S2, Interrupt Real-time Forwarding Architecture: Optimize the interrupt forwarding path through two-layer interrupt priority mapping, direct injection architecture, and interrupt isolation optimization; Step S3, vCPU scheduling real-time optimization: Improve vCPU scheduling determinism by configuring real-time scheduling strategies, strong affinity binding and high-precision wake-up methods; Step S4, Low latency in memory virtualization: Optimize memory virtualization latency by combining large page pre-allocation and locking, memory lock and TLB refresh optimization, and DMA lock granularity optimization.

[0013] In step S1, the critical section is compressed by removing non-atomic operations from the critical section in arch / arm64 / kvm / arm.c and arch / arm64 / kvm / vgic.c, leaving only the core logic of CPU instruction interaction under the protection of rt_mutex, thereby shortening the critical section duration.

[0014] Specifically, the preemption point insertion in step S1 is configured as follows: add exit type judgment logic; within the exception class function, for non-urgent exit types, insert a cond_resched_rt() preemption point; for urgent exit types, maintain atomic processing logic to avoid affecting system stability.

[0015] The dual-level interrupt priority mapping in step S2 includes: mapping the physical IRQ corresponding to the external device to a real-time interrupt thread of the SCHED_FIFO scheduling class using the rtirq tool, and assigning a priority of 60; at the same time, configuring the priority of the KVM-related interrupt thread to 70.

[0016] Specifically, the direct injection architecture in step S2 is configured as follows: the interrupt injection logic in the drivers / virt / kvm / irqfd.c file is modified to skip redundant intermediate forwarding steps; a priority mapping module is added to achieve linkage matching between the interrupt priority within the virtual machine and the interrupt thread priority of the host machine.

[0017] The interrupt isolation optimization method in step S2 is as follows: bind the real-time interrupt thread to a dedicated physical CPU core using the taskset tool, and disable the irqbalance service of the dedicated physical CPU core to prevent latency jitter caused by interrupt migration.

[0018] The real-time scheduling policy configuration in step S3 includes: configuring the qemu-kvm process of the corresponding vCPU as a SCHED_FIFO scheduling class and setting its priority to 40 via the command line; and limiting the CPU utilization of non-real-time virtual machines to no more than 50% through the cgroup CPU subsystem to prevent them from preempting high-priority vCPU resources.

[0019] In step S3, the strong affinity binding is set such that when the virtual machine starts, multiple vCPUs are bound to physical CPU cores respectively through the qemu-kvm parameter, and kernel.sched_migration_cost_ns=0 is configured in the Preempt_RT kernel to disable scheduling migration optimization.

[0020] The high-precision wake-up method in step S3 includes: modifying the kvm_vcpu_block() function in drivers / virt / kvm / kvm_main.c, replacing the traditional timer with hrtimer, setting the wake-up precision to the microsecond level, and reducing the latency of the vCPU blocking / wake-up process.

[0021] The memory lock and TLB refresh optimization in step S4 includes: replacing kvm_mmu_lock from spinlock to rt_mutex and enabling priority inheritance; modifying the TLB refresh logic in arch / arm64 / kvm / mmu.c to change full refresh to on-demand refresh, only refreshing the EPT page table entry corresponding to the current vCPU, thereby reducing TLB refresh overhead.

[0022] The beneficial effects of this invention are: The KVM real-time optimization method based on Preempt_RT of this invention reduces various core latencies of the KVM virtualization layer, effectively compresses latency jitter, and achieves high determinism and stability of real-time response in virtualization scenarios, which can meet the real-time requirements of virtualization technology in key fields such as industrial control. Attached Figure Description

[0023] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0025] The overall approach of this invention is to fully leverage the core real-time capabilities of the Preempt_RT kernel, such as full-path preemption, interrupt threading, and rt_mutex priority inheritance, and to perform targeted adaptation and collaborative optimization for the four core bottlenecks of KVM in the virtualization layer, interrupt forwarding, vCPU scheduling, and memory virtualization. Through four methods—fine-grained preemption optimization of the virtualization layer, real-time interrupt forwarding architecture, real-time optimization of vCPU scheduling, and low latency of memory virtualization—a three-level real-time enhancement architecture is constructed, consisting of "lower-level kernel support, intermediate virtualization enhancement, and upper-level scenario adaptation."

[0026] The specific approach is as follows: First, reconstruct the KVM virtualization layer locking mechanism and processing flow, achieving fine-grained preemption through lock replacement and preemption point insertion, thus shortening the preemption window. Second, based on interrupt threading technology, design a two-layer interrupt priority mapping and direct injection architecture to optimize interrupt forwarding paths. Third, improve vCPU scheduling determinism through configuring real-time scheduling policies, strong affinity binding, and high-precision wake-up. Fourth, optimize memory virtualization latency by combining technologies such as large page pre-allocation, real-time lock adaptation, and on-demand TLB refresh. Ultimately, this achieves precise control of microsecond-level latency and nanosecond-level jitter in KVM virtualization scenarios, meeting the real-time requirements of critical domains.

[0027] This invention is based on the Phytium D2000 hardware platform, and the software environment configuration is as follows: the host system uses the Linux 5.10.102-rt53 kernel (with the Preempt_RT real-time patch enabled), the virtualization layer uses KVM version 5.10, the virtual machine hypervisor is qemu-kvm 5.2.0, and the virtual machine operating system is Linux version 5.10 with the real-time patch; interrupt priority is configured using the rtirq tool, CPU affinity is configured using the taskset tool, and non-real-time task resource usage is limited using cgroups.

[0028] like Figure 1The diagram shows a flowchart of the KVM real-time optimization method based on Preempt_RT of the present invention. Fine-grained preemption optimization of the virtualization layer is achieved in step S1, the real-time interrupt forwarding architecture is achieved in step S2, real-time vCPU scheduling optimization is achieved in step S3, and low latency memory virtualization is achieved in step S4. The specific steps include steps S1 to S4 as described below.

[0029] Step S1, fine-grained preemption optimization of the virtualization layer: reconstruct the locking mechanism and processing flow of the KVM virtualization layer, and achieve fine-grained preemption by lock replacement, compression of critical sections and insertion of preemption points, thereby shortening the preemption window.

[0030] (1) The specific operation of lock replacement in step S1 is to modify the KVM core code and replace the core locks kvm_lock and vcpu->mutex from spinlock to Preempt_RT native rt_mutex.

[0031] (2) The compression critical region in step S1 is set as follows: In arch / arm64 / kvm / arm.c and arch / arm64 / kvm / vgic.c, non-atomic operations (such as log printing and parameter validation) in the VM-Exit, exception handling, and IO access processing flow are moved out of the critical section, and only the core logic such as CPU instruction interaction is retained under the protection of rt_mutex, thereby shortening the critical section duration.

[0032] (3) The setting of the insertion preemption point in step S1 is as follows: In arch / arm64 / kvm / arm.c, the exception handling and I / O access branch logic in the VM-Exit processing core were adjusted as follows: Exit type judgment logic was added. Within actual exception class functions such as kvm_vcpu_handle_exception() (exception class VM-Exit) and kvm_handle_io() (IO class VM-Exit), preemptive points of cond_resched_rt() were inserted for non-emergency exit types such as I / O port access and system register reads. For emergency exit types such as CPU exceptions and hardware errors, atomic processing logic was maintained to avoid affecting system stability.

[0033] Step S2, Interrupt Real-time Forwarding Architecture: Optimize the interrupt forwarding path through two-layer interrupt priority mapping, direct injection architecture, and interrupt isolation optimization.

[0034] (1) The dual-level interrupt priority mapping in step S2 includes: mapping the physical IRQs corresponding to external devices such as network cards and PCIe acquisition cards to real-time interrupt threads of the SCHED_FIFO scheduling class through the rtirq tool, and assigning a priority of 60; at the same time, configuring the priority of KVM-related interrupt threads (such as kvm-irqfd) to 70.

[0035] (2) The direct injection architecture in step S2 is set as follows: Modify the interrupt injection logic in the drivers / virt / kvm / irqfd.c file to construct the path of "physical interrupt threading → KVM_IRQFD event triggering → virtual machine MSI-X interrupt direct injection", skipping the redundant intermediate forwarding links; add a priority mapping module to realize the linkage matching between the interrupt priority in the virtual machine and the interrupt thread priority of the host.

[0036] (3) The interrupt isolation optimization method in step S2 is to bind the real-time interrupt thread to a dedicated physical CPU core through the taskset tool and disable the irqbalance service of the dedicated physical CPU core to prevent delay jitter caused by interrupt migration.

[0037] Step S3, vCPU scheduling real-time optimization: Improve vCPU scheduling determinism by configuring real-time scheduling strategies, strong affinity binding, and high-precision wake-up methods.

[0038] (1) The real-time scheduling policy configuration in step S3 includes: configuring the qemu-kvm process of the corresponding vCPU as the SCHED_FIFO scheduling class and setting the priority to 40 through the command line; limiting the CPU utilization of non-real-time virtual machines to no more than 50% through the cgroup cpu subsystem to prevent them from preempting high-priority vCPU resources.

[0039] (2) The strong affinity binding in step S3 is set to bind multiple vCPUs to physical CPU cores respectively through the qemu-kvm parameter when starting the virtual machine, that is, bind 4 vCPUs to physical CPU cores 0-3 respectively through the qemu-kvm parameter "-smp cores=4,threads=1,sockets=1-cpu host -affinity 0-3"; at the same time, configure kernel.sched_migration_cost_ns=0 in the Preempt_RT kernel to disable scheduling migration optimization.

[0040] (3) The high-precision wake-up method in step S3 includes: modifying the kvm_vcpu_block() function in drivers / virt / kvm / kvm_main.c, replacing the traditional timer with hrtimer, setting the wake-up precision to the microsecond level, and reducing the delay of the vCPU blocking / wake-up process.

[0041] Step S4, Low latency in memory virtualization: Optimize memory virtualization latency by combining large page pre-allocation and locking, memory lock and TLB refresh optimization, and DMA lock granularity optimization.

[0042] (1) The large page pre-allocation and locking method in step S4 is as follows: In the host kernel, configure vm.nr_hugepages=1024 (corresponding to 2MB large pages) through the sysctl command. When starting the virtual machine, add the parameter "-m 2048 -mem-prealloc-hugepages" to pre-allocate the virtual machine memory as 2MB large pages and lock it through mlockall to avoid memory swapping.

[0043] (2) The optimization method for memory lock and TLB refresh in step S4 is as follows: replace kvm_mmu_lock from spinlock to rt_mutex and enable priority inheritance. Modify the TLB refresh logic in arch / arm64 / kvm / mmu.c, change the full refresh to on-demand refresh, and only refresh the EPT page table entry corresponding to the current vCPU to reduce TLB refresh overhead.

[0044] (3) The method for optimizing the DMA lock granularity in step S4 is to optimize the kvm_dma_map() function, adopt a segmented lock mechanism, reduce the lock holding time, and ensure that DMA transfer and memory access are executed in parallel.

[0045] This invention replaces the KVM core lock from a spinlock with a native Preempt_RT rt_mutex, and solves priority inversion by combining a priority inheritance mechanism. It dynamically inserts preemption points according to the exit type in the VM-Exit processing path, achieving full lifecycle preemptibility of the virtualization core path, overcoming the design limitations of traditional KVM's "disable preemption for VM-Exit processing." By constructing a two-layer mechanism of "host interrupt threading + virtual machine interrupt priority mapping," it maps physical interrupts to high-priority real-time threads, optimizes the MSI-X interrupt direct injection logic, and achieves superior performance between physical and virtual machine interrupts. Priority linkage is implemented to resolve the "priority loss" problem in interrupt forwarding. A "real-time scheduling strategy + strong CPU affinity binding + high-precision wake-up" scheme is adopted, configuring vCPU processes as SCHED_FIFO scheduling classes and assigning them high priority, binding them to dedicated physical CPU cores, and achieving microsecond-level precision wake-up through hrtimer, avoiding vCPU migration and resource contention. By integrating "large page pre-allocation locking + memory access rt_mutex adaptation + on-demand TLB refresh" technologies, page table hierarchy and TLB refresh overhead are reduced, resolving memory lock contention and fragmentation issues, and achieving deterministic control of memory access latency. These technologies are used to improve the real-time performance of KVM virtual machines under Preempt_RT.

[0046] This invention builds a test environment based on a specified version of the Linux kernel with the Preempt_RT real-time patch and KVM. The technical solution of this invention is compared with the traditional KVM solution in performance testing. This invention reduces various core latencies of the KVM virtualization layer, effectively compresses the latency jitter, and achieves high determinism and stability of real-time response in virtualization scenarios, which can meet the real-time requirements of virtualization technology in key fields such as industrial control.

[0047] While the present invention discloses preferred embodiments to achieve the above objectives, these are not intended to limit the structural features of the invention. Anyone skilled in the art should know that any easily conceived variations or modifications are possible within the technical spirit of the invention and are covered by the claims of the present invention.

Claims

1. A preempt_RT based KVM real-time optimization method, characterized in that, The four aspects of KVM in the virtualization layer, interrupt forwarding, vCPU scheduling and memory virtualization are directionally adapted and cooperatively optimized, and a three-level real-time enhancement architecture of bottom kernel support-intermediate virtualization enhancement-top scene adaptation is constructed through four means of virtualization layer fine-grained preemption optimization, interrupt real-time forwarding architecture, vCPU scheduling real-time optimization and memory virtualization low delay, wherein the virtualization layer fine-grained preemption optimization is realized through step S1, the interrupt real-time forwarding architecture is realized through step S2, the vCPU scheduling real-time optimization is realized through step S3, and the memory virtualization low delay is realized through step S4. In step S1, the virtualization layer fine-grained preemption optimization is performed: the KVM virtualization layer lock mechanism and processing flow are reconstructed, fine-grained preemption is realized through lock replacement, critical region compression and preemption point insertion, and the preemption window is shortened. In step S2, the interrupt real-time forwarding architecture is constructed: through double-layer interrupt priority mapping, direct injection architecture and interrupt isolation optimization, the interrupt forwarding path is optimized. In step S3, the vCPU scheduling real-time optimization is performed: through configuration of real-time scheduling strategy, strong affinity binding and high-precision wake-up means, the vCPU scheduling determinacy is improved. In step S4, the memory virtualization low delay is realized: combined with large page pre-allocation and locking, memory lock and TLB flush optimization and DMA lock granularity optimization, the memory virtualization delay is optimized.

2. The Preempt_RT based KVM real-time optimization method according to claim 1, wherein, In the step S1, the setting of the compressed critical region is that in arch / arm64 / kvm / arm.c and arch / arm64 / kvm / vgic.c, the non-atomic operation is moved out of the critical region, only the CPU instruction interaction core logic is kept under the protection of rt_mutex, and the critical region duration is shortened.

3. The Preempt_RT based KVM real-time optimization method of claim 2, wherein, In the step S1, the setting of the inserted preemption point is that: the exit type judgment logic is added, in the exception type function, for the non-emergency exit type, the cond_resched_rt() preemption point is inserted; for the emergency exit type, the atomic processing logic is kept to avoid affecting the system stability.

4. The Preempt_RT based KVM real-time optimization method of claim 1, wherein, In the step S2, the double-layer interrupt priority mapping includes: through the rtirq tool, the physical IRQ corresponding to the external device is mapped into a real-time interrupt thread of the SCHED_FIFO scheduling class, and the priority 60 is assigned; at the same time, the priority of the KVM related interrupt thread is configured as 70.

5. The Preempt_RT based KVM real-time optimization method of claim 4, wherein, In the step S2, the setting of the direct injection architecture is that: the interrupt injection logic in the drivers / virt / kvm / irqfd.c file is modified to skip the redundant intermediate forwarding link; a priority mapping module is newly added to realize the linkage matching of the virtual machine interrupt priority and the host interrupt thread priority.

6. The Preempt_RT based KVM real-time optimization method of claim 5, wherein, In the step S2, the interrupt isolation optimization means is that: through the taskset tool, the real-time interrupt thread is bound to the exclusive physical CPU core, the irqbalance service of the exclusive physical CPU core is disabled, and the delay jitter caused by interrupt migration is prevented.

7. The Preempt_RT based KVM real-time optimization method according to claim 1, wherein, The real-time scheduling strategy configuration in the step S3 comprises: configuring the qemu-kvm process corresponding to the vCPU as the SCHED_FIFO scheduling class through the command line, setting the priority to 40; limiting the CPU usage of the non-real-time virtual machine to not more than 50% through the cpu subsystem of the cgroup, to avoid its preemption of the high-priority vCPU resource.

8. The Preempt_RT based KVM real-time optimization method of claim 7, wherein, The strong affinity binding in the step S3 is set as follows: when starting the virtual machine, the multiple vCPUs are respectively bound to the physical CPU core through the qemu-kvm parameter, and the kernel.sched_migration_cost_ns=0 is configured in the Preempt_RT kernel, to close the scheduling migration optimization.

9. The Preempt_RT based KVM real-time optimization method of claim 8, wherein, The high-precision wakeup means in the step S3 comprises: modifying the kvm_vcpu_block() function in drivers / virt / kvm / kvm_main.c, replacing the traditional timer with the hrtimer, setting the wakeup precision to the microsecond level, and reducing the delay of the vCPU blocking / wakeup process.

10. The Preempt_RT based KVM real-time optimization method of claim 1, wherein, The memory lock and TLB flush optimization in the step S4 comprises: replacing the kvm_mmu_lock from the spinlock to the rt_mutex and enabling the priority inheritance, modifying the TLB flush logic in arch / arm64 / kvm / mmu.c, changing the full flush to the on-demand flush, and flushing only the EPT page table item corresponding to the current vCPU, to reduce the TLB flush overhead.

Images