Transparent virtual machine monitoring method for cloud security

Abstract

The invention discloses a transparent virtual machine monitoring method for cloud security. The method comprises the following steps: S1, activating and stopping an LBR and a PMU; s2, integrity verification and user space interaction; s3, performing hardware configuration and interrupt processing; s4, performing double mapping on the kernel cache and the user space; and S5, user space interaction and CFI verification are carried out. Real-time monitoring and integrity verification of the control flow of the kernel of the virtual machine are achieved by ingeniously utilizing the hardware characteristics of a modern processor, control flow hijacking attacks are effectively resisted on the premise that normal operation of the virtual machine is not affected, and the safety of the kernel of the virtual machine in the cloud computing environment is improved.

Description

Technical Field

[0001] This invention relates to the field of virtual machine kernel security protection in cloud computing environments, and more specifically, to a method for implementing control flow integrity protection for virtual machine kernels based on a combination of processor hardware features. By leveraging the hardware characteristics of modern processors, this method provides efficient, transparent, and non-intrusive control flow integrity protection for virtual machine kernels, aiming to address security threats such as control flow hijacking faced by virtual machine kernels in cloud computing scenarios. Background Art

[0002] The current state of cloud computing and virtual machine security: The widespread adoption of cloud computing has led to the migration of a large number of computing tasks to the cloud, with the Platform as a Service model gaining widespread adoption. In this model, infrastructure providers provide virtual machines (VMs) to customers, allowing them to process private data or share information through server programs within the VMs. However, the attack surface of cloud computing environments is broader than that of on-premises environments. With all kinds of private data uploaded and processed in the cloud, cloud users expect their remote VMs to be fully protected, even better than on-premises systems.

[0003] The importance of kernel security and the threats it faces: As the core of a computer system, the kernel's security is the cornerstone of the security of the entire system. Kernel vulnerabilities can cause serious problems such as memory corruption, information leakage, or denial of service (DoS), posing a huge threat to system security. Unfortunately, kernel vulnerabilities continue to be exposed. In 2022 alone, the number of vulnerabilities assigned CVE IDs exceeded 300. At the same time, researchers continue to discover new kernel vulnerabilities through techniques such as fuzz testing. Corruption of function pointers or return addresses in memory can lead to control flow hijacking, which malicious attackers can use to redirect control flow to kernel space to execute malicious code, or even build kernel-mode rootkits. Therefore, protecting the virtual machine kernel from control flow hijacking attacks has become a key issue that needs to be addressed urgently.

[0004] Limitations of Existing Control Flow Integrity Protection Schemes: To combat kernel control flow hijacking attacks, various mechanisms have been proposed, such as shadow stacks, layout randomization, and Control-Flow Integrity (CFI). CFI protects against hijacking attacks by requiring that the execution of control flow instructions conform to the control flow graph (CFG) generated during program compilation. However, most existing CFI schemes are designed for on-premises environments and are difficult to directly apply in cloud computing due to the "separation of ownership." Specifically, cloud platform providers control the physical machines and have the highest hypervisor privileges, while cloud users fully control their VM guest operating systems. This prevents cloud users from leveraging VM hypervisor or hardware features to monitor their VMs, and cloud providers from tampering with the guest kernel to insert security check code. Furthermore, some software-instrumentation-based CFI schemes require modifying kernel code, potentially compromising kernel integrity and stability. Compatibility issues are prominent when working with a wide variety of operating systems and kernel modules, severely limiting the practicality of these schemes.

[0005] The potential and challenges of hardware-assisted solutions: Modern Intel x86 CPUs provide several hardware features for program debugging and system analysis, such as Branch Trace Store (BTS), Last Branch Record (LBR), and Performance Monitoring Unit (PMU), offering new approaches to addressing kernel security issues in cloud computing environments. However, each of these hardware features has limitations. BTS does not support instruction type filtering and records all branch instructions, resulting in high performance overhead. While LBR supports instruction type filtering and has low recording overhead, it cannot notify the CPU when the buffer is full. The PMU can monitor a variety of hardware events, but cannot independently record control flow effectively. Therefore, the present invention requires an innovative method to effectively combine these hardware features to achieve effective protection of the control flow integrity of virtual machine kernels in cloud computing environments. Summary of the Invention

[0006] The present invention aims to overcome the shortcomings of existing technologies by providing a method for protecting the control flow integrity of virtual machine kernels based on a combination of processor hardware features, thereby resolving the challenge of protecting the control flow integrity of virtual machine kernels in cloud computing environments. By cleverly utilizing the hardware characteristics of modern processors, this method enables real-time monitoring and integrity verification of the virtual machine kernel control flow. This method effectively defends against control flow hijacking attacks without affecting the normal operation of the virtual machine, thereby improving the security of the virtual machine kernel in cloud computing environments.

[0007] To achieve the above objectives, the present invention adopts a technical solution: a cloud security-oriented transparent virtual machine monitoring method, comprising:

[0008] S1. Activation and deactivation of LBR and PMU: Add probes to the KVM module to activate or deactivate LBR and PMU when the protected virtual machine VM is scheduled to enter or exit.

[0009] S2. Integrity Verification and Userspace Interaction: The host userspace code (code residing in the host machine) undergoes integrity verification. The kernel driver module KvmMon exports device files to the userspace as operation handles for interaction between the userspace code and the kernel.

[0010] S3, hardware configuration and interrupt handling: configure the control, data and status registers of LBR and PMU by setting up the VMCS hardware structure;

[0011] S4. Kernel cache and user space dual mapping: KvmMon uses a kernel memory buffer to cache all ICFTransfer execution records and "double maps" this buffer to user space;

[0012] S5. User space interaction and CFI verification: The prototype system exports device files to the host user space. The user space code interacts with the KvmMon module through customized mmap(), close(), and ioctl() system calls to achieve double mapping and release of the kernel cache buffer and communication with the KvmMon driver.

[0013] The prototype system is implemented by extending the KVM hypervisor on the Linux kernel. Its functions include driving the LBR and PMU features implemented in the KvmMon module, as well as the integrity checker in the host user space. The KvmMon module is responsible for managing the configuration and data collection of hardware features, while the user space program is responsible for executing the specific integrity verification logic.

[0014] When configuring LBR, instruction type filtering is implemented by configuring the MSR_LBR_SELECT register so that only indirect control flow transfers within the guest kernel space are recorded; the PMU's related PMC counter is increased each time a record is inserted into the LBR buffer. By initializing the PMC to the complement of the LBR stack size, it is ensured that the PMC overflows when the LBR buffer is full; at each VMEnter, KvmMon initializes and clears the related MSR registers of the LBR and PMU by setting the VMCS structure to ensure the correct operation of the hardware features.

[0015] The original KVM module is minimally modified, adding only multiple probes, where probes refer to function pointers. When the KvmMon module is loaded, the function pointers kvm_lbrpmu_setter() and kvm_lbrpmu_dumper() are initialized. When the vCPU is in the monitoring state, before the VM enters, the kvm_lbrpmu_setter() function will be called to set the VMCS hardware structure and enable the LBR and PMU functions. When the VM exits, the kvm_lbrpmu_dumper() function will be called to dump the indirect control flow transfer record (ICFTransfer) in the LBR buffer to the buffer.

[0016] During the initialization phase, the KvmMon module uses memory to create a cache buffer pool; each cache buffer consists of several consecutive memory pages, and the first page contains a header that manages the metadata of the current buffer; the buffer is used in a circular manner, and when the buffer overflows, the oldest record will be overwritten; through the API exported by the KvmMon module, the cache buffer and its header can be mapped to user space, and user space programs can read the ICFTransfer records in the buffer and obtain the buffer status by parsing the buffer header.

[0017] The prototype system exports device files to the host user space. The user space program implements dual mapping of the kernel cache buffer to the user space through a customized mmap() system call, and uses the ioctl() system call to implement communication between the user space code and the KvmMon module.

[0018] A user space program is used for integrity verification. The program adopts a multi-threaded approach and creates a working thread for each monitored vCPU for verification.

[0019] In step S3, the LBR records the execution of ICFTransfers in the guest kernel space, and the PMU monitors the record insertion events in the LBR buffer; when the LBR buffer is full, the PMU triggers a hardware interrupt, causing the VM to exit, and the VMExit handler checks the exit reason and dumps the LBR record to the cache buffer.

[0020] In step S5, the control flow graph (CFG) is extracted using the strategies of the CFIGuard and CFIMon schemes, and a tool chain is developed to organize the CFG into the expected structure. For each ICFTransfer record, if its source address can locate a valid ICFInstructions entry in the CFG and the target address is a member of the corresponding allowed set, the record is valid.

[0021] The advantage of this invention lies in solving the problem of protecting the integrity of virtual machine kernel control flows in cloud computing environments. By cleverly utilizing the hardware characteristics of modern processors, it enables real-time monitoring and integrity verification of virtual machine kernel control flows. This effectively defends against control flow hijacking attacks without affecting the normal operation of the virtual machine, thereby improving the security of virtual machines in cloud computing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The following is a brief description of the contents and symbols in the drawings of the present invention:

[0023] Figure 1 A system block diagram provided for an embodiment of the present invention.

[0024] Figure 2 This is a system workflow diagram provided by an embodiment of the present invention.

[0025] Figure 3 This is a performance evaluation diagram provided by an embodiment of the present invention, running an sftp server.

[0026] Figure 4 This is a performance evaluation diagram provided by an embodiment of the present invention, running an http server.

[0027] Figure 5 This is a performance evaluation diagram provided by an embodiment of the present invention, running the MySQL server.

[0028] Figure 6 Performance evaluation charts provided for embodiments of the present invention, running the Octane benchmark test suite in Windows and Linux virtual machines.

[0029] The English and Chinese versions of the technical terms in this plan are as follows:

[0030] VM: Virtual Machine

[0031] KVM: Kernel-based Virtual Machine, kernel-based virtual machine

[0032] LBR: Last Branch Record, last branch record

[0033] PMU: Performance Monitoring Unit

[0034] PMC: Performance Monitoring Counter, performance monitoring counter

[0035] MSR: Model Specific Register, specific model register

[0036] VMCS: Virtual Machine Control Structure, virtual machine control structure

[0037] CFI: Control-Flow Integrity

[0038] CFG: Control Flow Graph

[0039] KvmCFI: KVM-integrated Control-Flow Integrity, control flow integrity integrated with KVM

[0040] KvmMon: KVM-integrated control-flow Monitor, control flow monitor integrated with KVM

[0041] ICFTransfer: Indirect Control Flow Transfer, indirect control flow transfer;

[0042] VMEnter:Virtual Machine Enter, virtual machine enters

[0043] VMExit: Virtual Machine Exit, virtual machine exit DETAILED DESCRIPTION

[0044] The specific implementation of the present invention will be further explained in detail below by describing the best embodiment with reference to the accompanying drawings.

[0045] This invention aims to provide a method for protecting the integrity of virtual machine kernel control flows based on a combination of processor hardware features, addressing the challenge of protecting the integrity of virtual machine kernel control flows in cloud computing environments. By cleverly leveraging the hardware characteristics of modern processors, this method enables real-time monitoring and integrity verification of virtual machine kernel control flows. This method effectively defends against control flow hijacking attacks without impacting the normal operation of virtual machines, thereby improving the security of virtual machines in cloud computing environments.

[0046] By cleverly combining the hardware features of contemporary processors, a pure hardware approach is used to monitor the control flow generated by the virtual machine kernel during runtime, perform timely security checks, and transparently protect the virtual machine. The specific implementation steps are as follows:

[0047] S1: Add probes to the KVM module. These probes (probers) are triggered when a protected VM scheduler enters / exits, activating / deactivating the LBR and PMU. At the same time, integrity verification is allowed for code residing in the host machine, allowing cloud providers to flexibly select control flow integrity checking algorithms based on their needs without having to modify KvmCFI's kernel driver, KvmMon. KvmMon exports device files (in Linux, device files are special files that provide user-space programs with an interface for communicating with hardware devices or virtual devices) to user-space code (code residing in the host machine). User-space programs can interact with the KvmMon module through these device files. Specifically, user programs can open the device file to obtain a file handle, which they can then use as a parameter to invoke system calls for read and write operations. As described later, these system calls are customized and converted into controls for the KvmMon module.

[0048] S2: Leveraging the mechanisms provided by Intel CPUs and by setting up the VMCS hardware structure, a hypervisor (such as KVM) can configure the control, data, and status registers of the LBR and PMU for protected guests. The LBR is configured to record the execution of indirect control flow transfers (ICFTransfer) in guest kernel space. For each execution, a pair of valid addresses (jump source address, jump target address) is recorded in the LBR buffer. The PMU monitors record insertion events in the LBR buffer, and its associated PMC increments upon each record insertion event. By initializing the PMC to the complement of the LBR stack size, the PMC overflows when the LBR buffer is full. At this point, the PMU triggers a hardware interrupt, causing the VM to exit. The VMExit handler checks the exit reason. If the VM exits due to a PMC overflow, the KvmMon module dumps the LBR record to the cache buffer. In addition, to ensure security, the control registers (CR) that configure the LBR and PMU are not allowed to be accessed by the guest VM. Specific fields of the VMCS are set through the "vmread" and "vmwrite" instructions to make these registers invisible to the guest VM.

[0049] S3: Add probes to the KVM vCPU scheduling function vmx_vcpu_enter_exit() to receive notifications of VMEnter and VMExit events. Each probe is associated with a function pointer. When the KvmMon module is loaded, these function pointers will point to functions in KvmMon. Furthermore, a substructure named "cfi_ctrl" is added to the structure named "vcpu_vmx" associated with the KVM configuration to track the CFI-related status of a given vCPU. A flag in cfi_ctrl indicates whether the vCPU is in the monitoring state and determines whether the probe function is called. When the vCPU is in the monitoring state, before the VM enters, the function pointed to by the corresponding function pointer is called to set up the VMCS hardware structure and enable the LBR and PMU functions. When the VM exits, another function pointed to by another function pointer is called to dump the ICFTransfer records in the LBR buffer to the cache buffer.

[0050] S4: To achieve secure and flexible integrity verification, KvmCFI offloads integrity verification tasks to host userspace code. KvmMon uses a kernel memory buffer to cache all ICFTransfer execution records and, utilizing a "double mapping" mechanism, maps this buffer into userspace (this allows the contents of the same physical memory to be visible to both kernel code and user programs). Upon each VM exit, KvmMon dumps the LBR records into the cache buffer, even if the VM exit was not due to a full LBR buffer. For guest VMs with multiple vCPUs, KvmMon maintains a kernel cache buffer pool, with each monitored vCPU having its own cache buffer. The cache buffer is initialized when its associated vCPU first executes and is released when the vCPU stops monitoring. The buffer pool is created and destroyed when the KvmMon module is loaded and unloaded, respectively. Using the double mapping mechanism provided by the Linux kernel, KvmMon exports the cache buffer to host userspace code. Userspace programs can efficiently access ICFTransfer records by initiating double mapping requests on device files using the mmap() system call. mmap() is an important system call in the Linux operating system. Its main function is to map the contents of a file to the virtual address space of a process. As described below, this patent customizes its function to enable dual mapping.

[0051] S5: The prototype system (this patent implements a preliminary and simplified system based on KVM based on the design principle) exports device files to the host user space code, and the host user space code can interact with the KvmMon module through system calls. The customized mmap() system call implements double mapping of the kernel cache buffer to the user space, mapping the cache buffer as a read-only memory area. The customized close() system call is used to release the cache buffer and close the file. The customized ioctl() system call is used to establish a communication channel between the host user space code and the KvmMon driver. The host user space code implements related functions through the customized mmap() system and ioctl() system. In the Linux kernel, ioctl() allows user space code to operate devices through custom commands. By formulating a communication protocol, the user space code can send information to the KvmMon driver or retrieve data from it, such as querying the number of vCPUs and the kernel cache buffer size of the protected VM.

[0052] S6: The CFI scheme verifies integrity by referring to a given CFG, and the accuracy of the CFG is crucial to the effectiveness of the scheme. KvmCFI supports modularity, and its CFG contains cross-module nodes. The present invention adopts the strategies of CFIGuard and CFIMon to extract CFG, and develops a tool chain to organize the CFG into the expected structure. Specifically, like CFIGuard and CFIMon, static program analysis and dynamic runtime profiling are used to obtain a complete CFG control flow graph. Each CFG stores offset values, which are relative addresses of code points in the current module. When loading the CFG file for CFI verification, all offset values will be relocated and set to the real (fixed) address of the referenced code point. The CFG is organized into a two-level tree structure. The first level stores the addresses of all valid ICFInstructions, and the second level stores the allowed jump target addresses of each ICFInstruction. For each ICFTransfer record, if its source address can locate a valid ICFInstruction entry in the CFG and the target address is a member of the corresponding allowed set, the record is valid. For cross-module transfers, where the source address and target address are located in different modules, verification must ensure that the source address is valid in the current CFG and that the target address is a valid entry in the target CFG.

[0053] In the above design, the interaction process between each module has been carefully optimized to ensure efficient data transmission and processing. For example, in the interaction between the KVM probe and the kernel cache buffer, the monitoring data can be quickly and accurately stored.

[0054] When the KVM probe is triggered, it will first check the status of the vCPU to ensure the accuracy and safety of the operation and avoid the impact of erroneous operations on the system.

[0055] Regarding the kernel cache buffer, its size is dynamically adjusted based on the scale of the protected VM and the number of vCPUs to achieve reasonable resource utilization.

[0056] The user space API was designed with full consideration of compatibility and scalability, and can be easily integrated with different types of user space programs.

[0057] During the integrity verification process, an incremental update strategy is adopted for CFG updates to improve verification efficiency and reduce resource consumption.

[0058] When the LBR buffer is full, the system will use a fast dump mechanism to ensure that data is not lost, and at the same time reasonably schedule subsequent monitoring work.

[0059] In the collaborative work between the KVM probe and the LBR+PMU module, a synchronization mechanism is added to ensure data consistency.

[0060] When an exception occurs in the double mapping of the kernel cache buffer, the system will automatically perform a remapping operation to ensure that the user space program has access to the data.

[0061] like Figure 1 The figure shows the system architecture diagram corresponding to the monitoring method of this application. In a cloud computing environment, the kernel of the host system integrates the KVM module, which provides convenient support for creating and running virtual machines. The control flow protection solution proposed in this patent consists of two parts in terms of system implementation, both of which run in the host machine. One part of the functions is integrated into the LBR+PMU kernel module, which is mainly responsible for controlling and managing the LBR and PMU. The other part works in the user mode of the host machine and is mainly responsible for control flow integrity checking. When the virtual machine is started and running, the LBR+PMU module tracks the control flow transfer running in the virtual machine kernel and dumps the records to the kernel buffer. With the help of the "double mapping" mechanism, the kernel buffer will be mapped to the host user state. In this way, the host user state code can use CFG to perform accurate control flow integrity detection on the above transfer records, thereby effectively ensuring the security of the virtual machine.

[0062] like Figure 2 The following is a system workflow diagram, which provides further technical details. After the virtual machine is booted, the host user-mode program uses the open() and close() system calls to communicate with the LBR+PMU module, thereby manipulating the operation of these two CPU features. Furthermore, the user-mode program can use the mmap() system call to map the user kernel buffer storing control flow transfer records to itself, thereby performing control flow integrity verification.

[0063] like Figure 3-6 As shown in the figure, the test results show that the system has minimal performance overhead when implementing control flow integrity protection for virtual machines. In common virtual machine configurations such as 1 to 8 vCPUs, the throughput of servers such as sftp, http, and mysql will only incur an overhead of about 1.5%. Figure 6 As shown in the figure, similar conclusions can be drawn no matter on a Windows virtual machine or a Linux virtual machine.

[0064] Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, they are all within the scope of protection of the present invention.

Claims

1. A transparent virtual machine monitoring method for cloud security, characterized by: include S1. Activation and deactivation of LBR and PMU: Add probes to the KVM module to activate or deactivate LBR and PMU when the protected virtual machine VM is scheduled to enter or exit. S2. Integrity verification and user space interaction: The host user space code performs integrity verification, and the kernel driver KvmMon exports device files to the user space for operation by the user space code; S3, hardware configuration and interrupt handling: configure the control, data and status registers of LBR and PMU by setting up the VMCS hardware structure; S4. Kernel cache and user space dual mapping: KvmMon uses a kernel memory buffer to cache all ICFTransfer execution records and "double maps" the buffer to user space; S5. User space interaction and CFI verification: The prototype system exports device files to the host user space. The user space code interacts with the KvmMon module through customized mmap(), close(), and ioctl() system calls to achieve double mapping and release of the kernel cache buffer and communication with the KvmMon driver.

2. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: The prototype system is implemented by extending the KVM hypervisor on the Linux kernel. Its functions include driving the LBR and PMU features implemented in the KvmMon module, as well as the integrity checker in the host user space. The KvmMon module is responsible for managing the configuration and data collection of hardware features, while the user space program is responsible for executing the specific integrity verification logic.

3. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: When configuring LBR, instruction type filtering is implemented by configuring the MSR_LBR_SELECT register so that only indirect control flow transfers within the guest kernel space are recorded; the PMU's related PMC counter is increased each time a record is inserted into the LBR buffer. By initializing the PMC to the complement of the LBR stack size, it is ensured that the PMC overflows when the LBR buffer is full; at each VMEnter, KvmMon initializes and clears the related MSR registers of the LBR and PMU by setting the VMCS structure to ensure the correct operation of the hardware features.

4. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: The original KVM module is minimally modified, adding only multiple probes, where probes refer to function pointers. When the KvmMon module is loaded, the function pointers kvm_lbrpmu_setter() and kvm_lbrpmu_dumper() are initialized. When the vCPU is in the monitoring state, before the VM enters, the kvm_lbrpmu_setter() function will be called to set the VMCS hardware structure and enable the LBR and PMU functions. When the VM exits, the kvm_lbrpmu_dumper() function will be called to dump the ICFTransfer records in the LBR buffer to the cache buffer.

5. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: During the initialization phase, the KvmMon module uses memory to create a cache buffer pool; each cache buffer consists of several consecutive physical memory pages, and the first page contains a header that manages the metadata of the current buffer; the buffer is used in a circular manner, and when the buffer overflows, the oldest record will be overwritten; through the API exported by the KvmMon module, the cache buffer and its header can be mapped to user space, and user space programs can read the ICFTransfer records in the buffer and obtain the buffer status by parsing the buffer header.

6. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: The prototype system exports device files to the host user space. The user space program implements dual mapping of the kernel cache buffer to the user space through a customized mmap() system call, and uses the ioctl() system call to implement communication between the user space code and the KvmMon module.

7. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: Integrity verification is performed using a user space program. The program adopts a multi-threaded approach and creates a working thread for each monitored vCPU for verification.

8. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: In step S3, the LBR records the execution of ICFTransfers in the guest kernel space, and the PMU monitors the record insertion events in the LBR buffer; when the LBR buffer is full, the PMU triggers a hardware interrupt, causing the VM to exit, and the VMExit handler checks the exit reason and dumps the LBR record to the cache buffer.

9. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that: In step S5, the control flow graph (CFG) is extracted using the strategies of CFIGuard and CFIMon, and a tool chain is developed to organize the CFG into the expected structure. For each ICFTransfer record, if its source address can locate a valid ICFInstructions entry in the CFG and the target address is a member of the corresponding allowed set, the record is valid.

Images