Resource control method and device

Abstract

The embodiment of the invention provides a resource control method and device, and relates to the technical field of virtualization, the method comprises the steps that IO equipment is directly connected to a secure virtual machine through an equipment management module located in a TEE and a communication module with the privilege level higher than that of the equipment management module, the equipment management module does not belong to a TCB, and simplification of the TCB is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of virtualization technology, and in particular, to a resource control method and device. Background Art

[0002] In recent years, with the continuous development of virtualization technology, various lightweight virtualization technologies have emerged, among which typical representatives include container and language runtime-based virtualization technology. At present, secure virtual machines (also called confidential virtual machines) running in a secure world (also described as a trusted execution environment) are still the main implementation solution for confidential computing at this stage.

[0003] Currently, the resource control scheme for secure virtual machines needs to be further optimized. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a resource control method and device. In this method, the IO device is directly connected to the secure virtual machine through a device management module located in the TEE and a communication module with a higher privilege level than the device management module, wherein the device management module does not belong to the TCB, ensuring the simplification of the TCB.

[0005] In one possible implementation, the present application provides a resource control method, which is applied to a resource control device, the device comprising: a device management module, a communication module, wherein both the device management module and the communication module run in a trusted execution environment (TEE), and the privilege level of the communication module is higher than the privilege level of the device management module. The method comprises: the device management module sends first information to the communication module, wherein the first information includes identification information indicating a first IO device; the communication module obtains a first physical address range of the first IO device based on the first information; the communication module associates the first physical address range with a first secure virtual machine, wherein the first secure virtual machine runs in the TEE, and the privilege level of the first secure virtual machine is lower than the privilege level of the communication module.

[0006] The resource control device may be a central processing unit (CPU), or any type of processor, etc. The resource control device may be implemented as a software module and / or a hardware module. The present application does not limit the implementation method of the resource control device. For ease of explanation, the resource control device is implemented as a CPU as an example.

[0007] Exemplarily, the processing module may be hardware in a central processing unit including one or more signal processing and / or application-specific integrated circuits. The processing module may be used to execute instructions. The program running in the CPU may optionally include a communication module and a device management module. When the CPU is running the program of any one of the above two modules, the processing module may execute instructions in the currently running module to implement the functions of the currently running module.

[0008] In order to enable a secure virtual machine (also known as a confidential virtual machine) running in a secure world (such as a TEE environment) to avoid memory copying between the non-secure world and the secure world when accessing IO devices, the present application may pass the IO device (here referred to as the first IO device) directly to the secure virtual machine.

[0009] In order to directly pass the IO device to the secure virtual machine in the secure world without increasing the complexity of the TCB, the present application adds a device management module at a privilege level lower than the privilege level of the TCB (for example, the communication module can be a module in the TCB), and implements a communication module used in conjunction with the device management module to ensure data isolation and data security between the secure virtual machine, the device management module, and the ordinary world through the communication module.

[0010] For example, the present application may improve the S-visor to adapt to the device management module, so as to add the function of the communication module in the S-visor. For the sake of convenience, the communication module may be represented by S-visor.

[0011] In a possible implementation, the device management module may run in a second secure virtual machine, and the second secure virtual machine has the same privilege level as a first secure virtual machine that needs to pass through the IO device.

[0012] In order to ensure the simplification of S-visor, this application does not deploy the device management function of the device management module in S-visor, but deploys it in the virtual machine of the secure world (here the second secure virtual machine). In this way, the device management module will not expand the TCB by running in the secure virtual machine, and on the other hand, it can provide device management functions by reusing a complete Linux to reduce the implementation complexity. Since SSrv-VM is only responsible for managing and configuring devices and does not participate in the data plane, the additional sinking and forwarding overhead brought about by this is acceptable.

[0013] For ease of explanation, a service virtual machine is used to represent the device management module.

[0014] In an embodiment of the present application, in order to realize the direct pass of IO devices to the running secure virtual machine (represented here by the first secure virtual machine), the service virtual machine can manage IO devices and can indicate the identification information (such as IPA, etc.) of the first IO device it manages to the S-visor. The S-visor maintains and saves some information, through which the accessible physical address range of the first IO device related to the device identification can be found. The information maintained by the above S-visor for finding the physical address range of the IO device can be, but is not limited to: a secondary page table. For example, here the S-visor can query the secondary page table belonging to the service virtual machine to determine the physical address range of the IO device corresponding to the identification of the IO device. Then, the S-visor can map the queried physical address range to the first secure virtual machine (represented by S-VM). The specific mapping method can be to associate the physical address range with the S-VM, so that the device management module directly passes the IO device to the S-VM. After that, the S-VM can access the IO device in the secure world through the physical address range without having to make a memory copy between the non-secure world and the secure world.

[0015] In order to solve the two problems of adding management functions for security devices in N-visor, but N-visor belongs to the ordinary world and cannot access the security devices in the secure world, and S-visor is TCB, and in order to ensure TCB streamlining, the device management function cannot be added in S-visor, this application implements a device management module in the secure world of a privileged level layer lower than the TCB privilege level, so that direct configuration of IO devices can be performed in the secure world, and the device management module is not deployed in the S-visor, thereby not increasing the complexity of TCB.

[0016] For example, the present application adds a service virtual machine (such as the device management module) and adapts and improves the S-visor (such as the communication module). The service virtual machine can pass the device directly to the S-VM through the S-visor, so that the S-VM can directly access the IO device.

[0017] In a possible implementation, the resource control device also includes a processing module, and after the communication module associates the first physical address range with the first security virtual machine, the method also includes: the processing module queries the first physical address range associated with the first security virtual machine according to the first access address, and determines a second access address mapped to the first access address; wherein the first access address is a resource access address in an instruction currently running by the first security virtual machine, and the second access address is a physical address in the first physical address range; and the processing module accesses the target resource in the first IO device according to the second access address.

[0018] After the first IO device is directly passed to the S-VM, the S-VM can directly access the first IO device.

[0019] Specifically, when the S-VM accesses the target resources of the IO device, the processing module can run the instructions of the S-VM to query the physical address range of the IO device mapped to the S-VM by the communication module according to the first access address (for example, IPA) indicated by the instruction (for example, query the secondary page table belonging to the S-VM) to determine the physical address mapped to the IPA, such as FPA. Then, the processing module can access the target resources in the first IO device directly passed to the S-VM according to the FPA, thereby realizing the S-VM's access to the direct IO device without the need for memory copying between the secure world and the non-secure world.

[0020] In a possible implementation, the device management module includes identification information of at least one second IO device, and the at least one second IO device includes the first IO device.

[0021] The device management module maintains identification information of one or more IO devices, which are IO devices that have been directly connected to the device management module, including the first IO device that the device management module directly connects to the S-VM.

[0022] In other words, the device management module can only pass through the IO devices that it manages but are not directly passed through to other S-VMs, to the S-VM that currently needs the direct-pass device, to ensure the isolation of the IO devices that are directly passed through between different S-VMs, to ensure secure access to the IO devices, and to avoid cross-access of different S-VMs to the same IO device, which may lead to data leakage of S-VM tenants.

[0023] In a possible implementation, before the device management module sends the first information to the communication module, the method further includes: the communication module associating the second physical address range of the at least one second IO device with the device management module.

[0024] The reason why the device management module can pass the first IO device directly to the S-VM is that the first IO device needs to be an IO device that has been directly passed to the device management module. Therefore, before the device management module passes the first IO device directly to the S-VM, the communication module (such as S-visor) can pass some IO devices (the second IO device described here) directly to the device management module according to the instructions of N-visor, specifically by mapping the physical address ranges of these IO devices to the device management module, for example, the communication module can associate the physical address ranges of these IO devices with the device management module.

[0025] Exemplarily, the specific mapping method may be that the S-visor configures a two-stage page table belonging to the device management module (eg, a service virtual machine), and the two-stage page table includes the physical address range of the second IO device.

[0026] In this way, the device management module can pre-connect at least one IO device, so that the IO device that has been connected can be switched to a certain S-VM to connect the S-VM device directly, so as to achieve the connection of the S-VM device.

[0027] In a possible implementation, the communication module obtains the first physical address range of the first IO device based on the first information, including: the communication module obtains the first physical address range of the first IO device within the second physical address range associated with the device management module based on the first information, wherein the second physical address range includes the first physical address range.

[0028] Among them, when the communication module receives the first information and directly passes the IO device to the S-VM, the first physical address range of the first IO device can be within the second physical address range of the IO device mapped to the service virtual machine. For the IO device not within the second physical address range, it means that it is not the IO device that has been directly passed to the service virtual machine. The communication module will not switch the IO device that is not directly passed to the service virtual machine to be directly passed to the S-VM, but will only search for the physical address range corresponding to the first IO device within the physical address range of the IO device directly passed to the service virtual machine to map it to the S-VM, thereby achieving the purpose of directly passing the first IO device to the S-VM and ensuring secure access to the IO device.

[0029] In a possible implementation, after the communication module acquires the first physical address range of the first IO device based on the first information, the method further includes: the communication module canceling the association between the first physical address range and the second secure virtual machine.

[0030] Since the physical address range of the first IO device has been mapped to the S-VM, so that the first IO device is directly passed to the S-VM, in order to ensure that the first IO device can only be accessed by the directly passed S-VM and cannot be accessed by the service virtual machine, the S-visor can cancel the association between the first physical address range of the first IO device and the service virtual machine to cancel the direct relationship between the first IO device and the service virtual machine.

[0031] In a possible implementation, the method further includes: the device management module sends a first request to the communication module; before the communication module issues a second request, the communication module hides the value of the register of the resource control device; wherein the second request is a request after the communication module processes the first request.

[0032] In order to protect the data in the service virtual machine (device management module) from being attacked and leaked, a communication module can be applied to the TCB (e.g., S-visor). Before the communication module sends a request from the device management module, for example, before forwarding the second request to the N-visor or S-VM, the value of the register of the resource control device (e.g., the register of the processing module) can be hidden (the register value can be deleted or hidden in a modified manner), thereby achieving the purpose of register field protection. In this way, before the code executed by the processing module switches from the code of the service virtual machine to the code of the N-visor or S-VM, the register field of the processing module (which may include relevant data of the code of the service virtual machine) can be cleared and hidden. This ensures data isolation between the service virtual machine and the S-VM, and data isolation between the service virtual machine and the N-visor.

[0033] In a possible implementation, the method further includes: the communication module receiving a third request of the first secure virtual machine; before the communication module issues a fourth request, hiding the value of the register of the resource control device; wherein the fourth request is a request after the communication module processes the third request.

[0034] In order to protect the data in the security virtual machine (here represented by the first security virtual machine) from being attacked and leaked, a communication module applied to the TCB (such as S-visor) can be used. Before the communication module sends a request from the security virtual machine, for example, before forwarding the fourth request to the N-visor or the service virtual machine, the value of the register of the resource control device (such as the register of the processing module) can be hidden (the value of the register can be deleted, or modified, etc. to hide it), so as to achieve the purpose of register field protection. In this way, before the code executed by the processing module switches from the code of the security virtual machine to the code of the N-visor or the service virtual machine, the register field of the processing module (which may include relevant data of the code of the security virtual machine) can be cleared and hidden. The data isolation between the security virtual machine and the service virtual machine, and the data isolation between the security virtual machine and the N-visor is ensured.

[0035] In a possible implementation, after the communication module associates the first physical address range with the first security virtual machine, the method further includes: the communication module determines, according to a third access address, that the first physical address range does not include a physical address mapped to the third access address by querying the first physical address range associated with the first security virtual machine; wherein the third access address is a resource access address in a first access request sent by the first security virtual machine to the communication module; and the communication module refuses to forward the first access request.

[0036] In order to ensure that the S-VM can only access IO devices that are directly passed to the S-VM, and cannot access IO devices that are not directly passed to the S-VM, the S-visor of the present application can, after receiving the first access request from the S-VM to a certain IO device, query whether there is an FPA corresponding to the IPA (an example of the third access address) in the first access request within the physical address range mapped to the S-VM. If not, the S-visor can refuse to forward the first access request to avoid the S-VM from illegally accessing the IO devices that are not directly passed through, thereby ensuring safe access to the IO devices.

[0037] The communication module of the present application can perform an address check on the device access request from the S-VM, and only allow it to access if it confirms that the address it accesses is within its access authority (for example, through a two-stage page table query), otherwise its access request will be rejected.

[0038] In a possible implementation, the number of the processing modules is at least one, and the processing module includes at least one group of first registers, each group of the first registers is used to record a third physical address range of a segment of memory resources, the organizational granularity of the memory resources, and at least one security attribute of the memory resources.

[0039] In a possible implementation, the organizational granularity is a region granularity or a page granularity, wherein the memory resources at the region granularity include at least one memory region, and the memory resources at the page granularity include at least one memory page.

[0040] In one possible implementation, the security attribute is a first attribute representing secure memory or a second attribute representing ordinary memory, wherein the memory resources with the security attribute being the first attribute are accessed only in the TEE environment, and the security attributes of each memory page in the page-granularity memory resources are independent of each other, and the security attributes of each memory area in the area-granularity memory resources are the same.

[0041] The resource control device may include one or more processing modules. Taking the resource control device as a multi-core CPU as an example, the resource control device may include multiple processing modules.

[0042] Any processing module may include at least one set of first registers (also called memory protection registers, or per-cpu registers).

[0043] A set of memory protection registers is used to record a continuous physical address range of memory resources.

[0044] A CPU may have at least one set of per-cpu registers, wherein the set of per-cpu registers may be used to represent a memory address range, an organizational granularity (also called a protection mode), and at least one security attribute.

[0045] The CPU can have one or more sets of per-cpu registers. Each set of per-cpu registers can record a memory range. The memory range defined by multiple sets of per-cpu is the memory range accessible to the CPU.

[0046] For example, a group of per-cpu registers may include at least 4 per-cpu registers, and at least 4 per-cpu registers are used to record different attributes of a memory address range. The attributes are the base address of the memory area (referred to as Base attribute), the length of the memory area (referred to as Len attribute), the organizational granularity of the memory area (referred to as Mod attribute), and the security attribute of the memory area (referred to as Attr attribute).

[0047] Among them, the per-cpu register recording the Base attribute can record the base address of the memory address range;

[0048] The per-cpu register that records the Len attribute can record the length of the memory address range;

[0049] Then the per-cpu register that records the Base attribute and the per-cpu register that records the Len attribute can jointly record a memory address range.

[0050] The per-cpu register that records the Attr attribute may record the attributes of secure memory (for example, represented by Secure) or normal memory (for example, represented by normal).

[0051] The per-cpu register recording the Mod attribute can record whether the organizational granularity of the memory address range is a regional granularity (eg, Region) or a page granularity (eg, Page).

[0052] The memory address range of Region granularity may include one or more memory regions, and each memory region may include one or more memory pages.

[0053] In one scenario, when the Mod of a memory address range represented by a per-cpu register is Region granularity, and the value of the per-cpu register representing the record Attr attribute of the memory address range is Secure, then the memory address range represented by this group of per-cpu registers is secure memory.

[0054] In another scenario, when the Mod of a memory address range represented by a per-cpu register is Region granularity, and the value of the per-cpu register representing the record Attr attribute of the memory address range is normal, then the memory address range represented by this group of per-cpu registers is all non-secure memory (also called normal memory).

[0055] The memory address range of page granularity may include one or more memory pages.

[0056] When the Mod of a memory address range represented by a set of per-cpu registers is Page granularity, the number of per-cpu registers recording the Attr attribute can be multiple, and each per-cpu register here is used to record the security attributes (secure memory or normal memory) of different pages in the memory address range.

[0057] For example, in one scenario, when the Mod of a memory address range represented by a group of per-cpu registers is Page granularity, for example, the memory address range includes 3 memory pages, then the group of per-cpu registers may include 3 per-cpu registers for recording the Attr attribute, where the value of each per-cpu register for recording the Attr attribute may be Secure or normal, so that the 3 per-cpu registers for recording the Attr attribute are respectively used to record whether the security attribute of each of the above 3 memory pages is secure memory or normal memory.

[0058] In this way, when the protection mode of a memory address range represented by a set of per-cpu registers is Page granularity, each memory page in the memory address range can have its own security attribute (Attr attribute), and its security attribute can be Secure or normal, so that each memory page can be secure memory or non-secure memory. Then when the protection mode of a memory address range represented by a set of per-cpu registers is Page granularity, the memory address range represented by the set of per-cpu registers can include any number of memory pages belonging to secure memory, so that the number of secure memories accessed by confidential virtual machines is not limited to 8 from the hardware through the per-cpu registers of the CPU. In this way, the limitation problem that TZASC has a maximum of 8 Memory Regions can be solved at the hardware level, and the purpose of increasing the upper limit of the number of confidential virtual machines as much as possible under the premise of pinning memory is achieved.

[0059] In the embodiment of the present application, the memory area of ​​the secure memory is classified into Page granularity and Region granularity. The secure memory of Page granularity is a single memory page, which can minimize memory fragmentation and improve resource utilization. The secure memory of Region granularity can cover multiple memory pages. By accessing the secure memory of Region type, the confidential virtual machine can access multiple memory pages at one time, thereby reducing the access overhead of the secure memory.

[0060] In this way, the embodiment of the present application manages the secure memory accessed by the secure world through a hybrid method of Page granularity and Region granularity, thereby achieving a balance between memory access efficiency and memory utilization.

[0061] In a possible implementation, the method further includes: the processing module configuring the value of the at least one group of first registers according to a first instruction; wherein the first instruction is the instruction currently executed by the communication module.

[0062] This application can prevent the configuration of secure memory from being tampered with by limiting the privilege level of memory access. For example, this application only allows code running in S-EL2 (such as the communication module in the S-visor) to write per-cpu registers.

[0063] In the embodiment of the present application, a program, here a communication module, running in the TEE at a higher privilege level than the secure virtual machine configures the value of the register to limit the secure memory accessible in the TEE.

[0064] In a possible implementation, the method further includes: the communication module associates the first physical address of the first memory page with information indicating that the security attribute of the memory page is the first attribute; wherein the first memory page is a memory page associated with the first physical address and the first security virtual machine (for example, the two-stage page table of the memory device belonging to the first security virtual machine S-VM includes a mapping of the IPA and HPA of the first memory page); the processing module modifies the value of the second register in the first register to a value indicating the first attribute according to the second instruction; wherein the second register is the first register in the group of first registers for recording the security attribute of the first memory page; wherein the second instruction is the instruction currently executed by the communication module.

[0065] When allocating Page-level secure memory to the S-VM, for example, allocating the first memory page to the S-VM, the first physical address of the first memory page can be associated with the security attribute of the first memory page as secure memory (for example, an example of the first attribute) by configuring the three-stage page table of the memory page. After the three-stage page table is configured for the first memory page allocated to the S-VM, the value of the memory protection register (the second register here) used to record the security attribute of the memory page in the memory protection register of the processing module needs to be changed from non-secure memory (or ordinary memory) to secure memory.

[0066] In a possible implementation, the method also includes: the communication module associates the second physical address of the second memory page with the first secure virtual machine; the processing module modifies the value of the third register in the first register to a value indicating the first attribute according to the third instruction; wherein the third register is the first register in the group of first registers for recording the security attribute of the first memory area, wherein the first memory area includes the second memory page; wherein the third instruction is the instruction currently running by the communication module.

[0067] In an embodiment of the present application, when allocating region-granular secure memory to the S-VM, there is no need to configure a three-level page table. It is only necessary to allocate a memory page to the S-VM and adjust the security attributes of the memory protection register of the memory region to which the memory page belongs to secure memory.

[0068] In a possible implementation, the present application provides a resource control device. The resource control device includes: a device management module and a communication module, wherein both the device management module and the communication module run in a trusted execution environment TEE, and the privilege level of the communication module is higher than the privilege level of the device management module; the device management module is used to send first information to the communication module, wherein the first information includes identification information indicating a first IO device; the communication module is used to obtain a first physical address range of the first IO device based on the first information; the communication module is used to associate the first physical address range with a first secure virtual machine, wherein the first secure virtual machine runs in the TEE, and the privilege level of the first secure virtual machine is lower than the privilege level of the communication module.

[0069] In a possible implementation, the device management module runs in a second secure virtual machine.

[0070] In a possible implementation, the resource control device also includes a processing module; the processing module is used to query the first physical address range associated with the first security virtual machine according to the first access address, and determine the second access address mapped to the first access address; wherein the first access address is the resource access address in the instruction currently running by the first security virtual machine, and the second access address is a physical address in the first physical address range; the processing module is also used to access the target resource in the first IO device according to the second access address.

[0071] In a possible implementation, the device management module includes identification information of at least one second IO device, and the at least one second IO device includes the first IO device.

[0072] In a possible implementation, the communication module is further configured to associate the second physical address range of the at least one second IO device with the device management module.

[0073] In a possible implementation, the communication module is specifically used to obtain a first physical address range of the first IO device within the second physical address range associated with the device management module based on the first information, wherein the second physical address range includes the first physical address range.

[0074] In a possible implementation, the communication module is further configured to cancel the association between the first physical address range and the second secure virtual machine.

[0075] In a possible implementation, the device management module is further used to send a first request to the communication module; the communication module is further used to hide the value of the register of the resource control device before issuing a second request; wherein the second request is a request after the communication module processes the first request.

[0076] In a possible implementation, the communication module is also used to receive a third request from the first secure virtual machine; the communication module is also used to hide the value of the register of the resource control device before issuing a fourth request; wherein the fourth request is a request after the communication module processes the third request.

[0077] In a possible implementation, the communication module is further used to determine, according to a third access address, by querying the first physical address range associated with the first security virtual machine, that the first physical address range does not include a physical address mapped to the third access address; wherein the third access address is the resource access address in the first access request sent by the first security virtual machine to the communication module; and the communication module is further used to refuse to forward the first access request.

[0078] In a possible implementation, the number of the processing modules is at least one, and the processing module includes at least one group of first registers, each group of the first registers being used to record a third physical address range of a segment of memory resources, the organizational granularity of the memory resources, and at least one security attribute of the memory resources.

[0079] In a possible implementation, the organizational granularity is a region granularity or a page granularity, wherein the memory resources at the region granularity include at least one memory region, and the memory resources at the page granularity include at least one memory page.

[0080] In one possible implementation, the security attribute is a first attribute representing secure memory or a second attribute representing ordinary memory, wherein the memory resources with the security attribute being the first attribute are accessed only in the TEE environment, and the security attributes of each memory page in the page-granularity memory resources are independent of each other, and the security attributes of each memory area in the area-granularity memory resources are the same.

[0081] In a possible implementation, the processing module is further configured to configure the value of the at least one group of first registers according to a first instruction; wherein the first instruction is the instruction currently executed by the communication module.

[0082] In one possible implementation, the communication module is further used to associate the first physical address of the first memory page with information indicating that the security attribute of the memory page is the first attribute; wherein the first memory page is a memory page in which the first physical address is associated with the first secure virtual machine; the processing module is further used to modify the value of the second register in the first register to a value indicating the first attribute according to a second instruction; wherein the second register is the first register in the group of first registers for recording the security attribute of the first memory page; wherein the second instruction is the instruction currently running by the communication module.

[0083] In one possible implementation, the communication module is further used to associate the second physical address of the second memory page with the first secure virtual machine; the processing module is further used to modify the value of the third register in the first register to a value indicating the first attribute according to a third instruction; wherein the third register is the first register in the group of first registers for recording the security attribute of the first memory area, wherein the first memory area includes the second memory page; wherein the third instruction is the instruction currently running by the communication module.

[0084] The effects of the resource control devices of the above embodiments are similar to the effects of the resource control methods of the above embodiments, and will not be described in detail here.

[0085] In a possible implementation, the present application provides a resource control device. The resource control device includes one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the processor can implement the method in any of the above implementations.

[0086] The effect of the resource control device of this embodiment is similar to the effect of the resource control method of the above embodiments, and will not be described in detail here.

[0087] In a possible implementation, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the method in any one of the above implementations.

[0088] The effect of the computer-readable storage medium of this embodiment is similar to the effect of the resource control method of the above embodiments, and will not be described in detail here.

[0089] In a possible implementation, the present application provides a computer program product. The computer program product includes a software program, and when the software program is executed by a computer or a processor, the method in any one of the above implementations is executed.

[0090] The effect of the computer program product of this embodiment is similar to the effect of the resource control method of the above embodiments, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0092] Figure 1 The system architecture diagram of TwinVisor for ARM platform is shown as an example;

[0093] Figure 2 The figure is a schematic diagram showing the network IO process of a common virtual machine and a confidential virtual machine.

[0094] Figure 3 A schematic diagram of a system architecture that supports secure device pass-through is shown as an example;

[0095] Figure 4a A schematic diagram of a processing process is shown as an example;

[0096] Figure 4b A schematic diagram of a processing process is shown as an example;

[0097] Figure 4c A schematic diagram of a processing process is shown as an example;

[0098] Figure 4d A schematic diagram of a processing process is shown as an example;

[0099] Figure 4e A schematic diagram of a processing process is shown as an example;

[0100] Figure 4f A schematic diagram showing an exemplary memory change;

[0101] Figure 5 A schematic diagram of the structure of a device provided in an embodiment of the present application;

[0102] Figure 6 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0103] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0104] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0105] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0106] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0107] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0108] In recent years, the application of cloud computing technology has become more and more widespread and popular. Enterprises and individual users can rent virtual machines provided by cloud platforms and build their own applications in them, saving the economic cost of purchasing hardware and maintenance. However, since the data is no longer stored on the user's personal computer or enterprise computer room, it is very important to ensure the confidentiality of the user's sensitive data on the cloud platform. Confidential computing is a cloud computing technology that isolates the user's sensitive data in a protected memory area, and any unauthorized code cannot access these protected data. With the widespread application of cloud computing, confidential computing is becoming more and more popular and gradually becoming a key technology for the future.

[0109] In recent years, with the continuous development of virtualization technology, various lightweight virtualization technologies have emerged one after another, among which typical representatives include containers and language runtime-based virtualization technologies. However, on the one hand, the isolation of these technologies has been questioned because they share the host kernel; on the other hand, as system virtualization continues to evolve, its performance overhead is gradually decreasing. Therefore, at this stage, major cloud vendors still use system virtualization as the main way to provide services to the outside world. Under the premise of balancing performance and security, confidential virtual machines (also called secure virtual machines) running in a secure world (also described as a trusted execution environment) are still the main implementation solution for confidential computing at this stage.

[0110] ARM TrustZone is a hardware security feature provided by the ARM platform. TrustZone introduces the Normal World and the Secure World at the CPU level, and each memory page has one of the above two attributes. TZASC (TrustZone Address Space Controller) is used to configure the security attributes of the memory in ARM. When the CPU is in the Normal World and attempts to access the memory of the Normal World, TZASC will trigger a page fault exception.

[0111] The secure world in this article can also be described as a trusted execution environment, and the normal world can also be described as a normal execution environment or a rich execution environment (REE).

[0112] Applications in the Normal World can fall through the SMC (Secure Monitor Call) to the S-EL3 privilege level, and then further forward to the Secure World. This operation is called a world switch. Similar to memory, devices also have the attributes of the Normal World or the Secure World. Code in the Normal World cannot access devices in the Secure World.

[0113] At the same time, ARM introduced the S-EL2 privilege level in v8.4 to provide hardware virtualization support for confidential virtual machines in Secure World. S-EL2 implements similar functions to the N-EL2 privilege level in Secure World. For example, VSTTBR_EL2 stores the second-stage page table in Secure World. When a confidential virtual machine accesses an intermediate physical address (IPA), the virtual machine manager of the secure world can translate the IPA into the mapping relationship of the host physical address (HPA) based on the second-stage page table.

[0114] Figure 1 The following is a schematic diagram of the system architecture of TwinVisor on the ARM platform.

[0115] The system architecture of TwinVisor is divided into an untrusted part in the normal world and a trusted part in the secure world. The normal world is divided into three privilege levels from low to high, namely N-EL0, N-EL1, and N-EL2. The secure world is also divided into three privilege levels from low to high, namely S-EL0, S-EL1, and S-EL2. In addition, the privilege level of the ARM platform also includes EL3, in which a monitor runs. The EL3 privilege level does not distinguish between the normal world and the secure world.

[0116] like Figure 1 As shown in the figure, TwinVisor runs two independent virtual machine managers (Hypervisors) in the normal world and the secure world at the same time, namely the Hypervisor running in the normal world (named N-visor) and the Hypervisor running in the secure world (named S-visor). In addition, the Hypervisor is also called a virtual machine monitor (VMM).

[0117] like Figure 1 As shown in the figure, N-visor runs at the N-EL2 privilege level and is responsible for resource management of the normal virtual machine (N-VM) and the confidential virtual machine (S-VM). S-visor runs at the S-EL2 privilege level. As the trusted computing base (TCB) of the entire TwinVisor system, S-visor can isolate N-visor and S-VM to prevent untrusted N-visor and S-VM from attacking each other.

[0118] In addition, if Figure 1As shown, both ordinary virtual machines and confidential virtual machines run on the host, and the host operating system also runs at the N-EL2 privilege level.

[0119] like Figure 1 As shown, no matter it is a normal virtual machine or a confidential virtual machine, they can use their own client operating systems to run application programs. For example, a normal virtual machine can use a client operating system (also called N-Guest OS) in the normal world at the N-EL1 privilege level to run application 11 and application 21 at the N-EL0 privilege level. A confidential virtual machine can use a client operating system (also called S-Guest OS) in the secure world at the S-EL1 privilege level to run application 12 and application 22 at the S-EL0 privilege level.

[0120] For a malicious hypervisor, on the one hand, it may read the memory of the virtual machine, thereby leaking sensitive data in the virtual machine; on the other hand, if the hypervisor can directly tamper with the memory of the virtual machine, it may cause abnormal behavior of the virtual machine. For example, it can forcibly change the control flow inside the virtual machine by modifying the return pointer at the bottom of the function stack, or cause an unrecoverable first-stage page fault exception by overwriting the pte inside the virtual machine. Therefore, for security reasons, the memory of confidential virtual machines has strict integrity protection requirements.

[0121] TwinVisor can use TrustZone to support the operation of confidential virtual machines. As mentioned above, TZASC (a type of hardware) can be used to configure the security attributes of memory in ARM. When the CPU is in Normal World and attempts to access the memory of Normal World, TZASC will trigger a page fault exception. However, the current implementation of TZASC only supports up to 8 memory regions for the secure world, so that confidential virtual machines can use up to 8 memory regions.

[0122] From the perspective of the operating system, it believes that as long as the virtual address space presented to the upper-level users is continuous, it is sufficient. As for whether the physical address of the memory is continuous, the user state does not need to care. This design idea is applicable to most scenarios. After all, in most cases, the program's need for memory is limited to reading and writing. As long as these two tasks can be completed smoothly, the upper-level application does not care about the actual physical distribution of memory pages. However, the secure memory used to support the operation of confidential virtual machines poses new challenges to this assumption of the operating system.

[0123] On ARM platforms that lack Realm Mode support, the integrity of the secure memory of confidential virtual machines depends on the secure world in TrustZone. However, since ARM's secure world only supports up to 8 Memory Regions, when confidential virtual machines need to use multiple non-contiguous physical memories, the 8 physically discrete Memory Regions will be quickly exhausted. After the Memory Region is exhausted, the Hypervisor will no longer be able to allocate secure memory, and will no longer be able to meet the needs of confidential virtual machines to dynamically expand memory or create confidential virtual machines, thereby limiting the upper limit of the number of confidential virtual machines that can run on a single physical machine.

[0124] To solve this problem, TwinVisor reuses the Contiguous Memory Allocator (CMA) in the existing Linux kernel and extends it to implement Split-CMA. Split-CMA can include the NormalWorld part and the Secure World part. When Linux starts, a memory pool will be reserved in advance for Split-CMA to use. In each memory pool, Split-CMA can use memory migration and merging methods to enable the buddy allocator in N-visor to obtain continuous physical memory in the secure world as much as possible, thereby solving the problem of limited Memory Region. In this way, TwinVisor can use memory migration and merging methods. When the memory allocator cannot find enough continuous memory, it can move the allocated memory pages to free up enough continuous memory space, thereby solving the problem caused by limited MemoryRegion.

[0125] Currently, I / O virtualization solutions mainly include para-virtualization (PV) I / O and device pass-through. However, PV IO will incur additional performance overhead in confidential virtual machine scenarios.

[0126] Figure 2 The figure is a schematic diagram showing the network IO process of a common virtual machine and a confidential virtual machine.

[0127] Take network I / O as an example. Figure 2 As shown, in the non-confidential virtualization scenario, refer to Figure 2In the ordinary virtual machine I / O process shown in the figure, if the application in the ordinary virtual machine needs to send a packet, the front-end driver (such as the virtual machine kernel) only needs to store the corresponding data address information into the virtual IO ring buffer (VirtIO Ring), and the back-end driver (such as the host kernel) finds the corresponding data through the metadata in the ring, and then calls the packet sending interface on the host to send the data out.

[0128] In the confidential virtualization scenario, refer to Figure 2 In the confidential virtual machine I / O process shown, the application in the confidential virtual machine needs to send a packet, and the corresponding data address information is stored in the secure memory of the confidential virtual machine. The secure memory is an encrypted memory, and the back-end driver (such as the host kernel) cannot directly access the secure memory. In order to solve this problem, the front-end driver can copy the data from the secure memory to the non-secure memory (also called normal memory) when establishing the DMA mapping, and then the back-end driver reads the data from the non-secure memory. Similarly, when receiving packets, the front-end driver is also required to copy the received IO data from the normal memory (also called non-secure memory) to the secure memory.

[0129] Then the extra data copy will bring serious performance overhead. In recent years, with the rapid growth of network card bandwidth, various high-performance network cards have emerged one after another, and the extra copy has a great impact on the performance of I / O intensive applications.

[0130] Therefore, it is increasingly important to adopt IO virtualization based on the secure device pass-through solution. By controlling the confidential virtual machine to directly access the secure device, on the one hand, the device pass-through solution saves the virtualization overhead compared to PV IO. On the other hand, since the secure device and the confidential virtual machine belong to the same Secure World, the secure device can directly access the secure memory of the confidential virtual machine, which naturally eliminates the additional memory copy overhead. The performance advantage of the device pass-through solution over the PV I / O solution is more prominent. More importantly, in recent years, SRIOV technology has gradually matured and various high-performance network cards have been deployed and widely used in data centers. So how to invest a low cost in the early stage can easily enable major cloud service providers to directly apply the device pass-through solution to their own business, which has become a problem that needs to be solved at present.

[0131] Currently, there are two main solutions for confidential virtualization on the ARM platform. One is the solution using TwinVisor mentioned above, and the other is ARM CCA using hardware extensions.

[0132] In order to achieve direct access to secure devices on the ARM platform, we take the above TwinVisor as an example. In order to solve the problem that TZASC only allows a maximum of 8 Memory Regions, TwinVisor uses memory migration and merging technology to ensure that Secure Memory is as continuous as possible. However, since existing mainstream devices do not support the processing of IOPF (related to the above), the existing device direct access solution still requires pinning memory to prevent the pages used by the device DMA from being swapped out by the operating system. TwinVisor's Split-CMA solution relies on dynamic memory migration. Then the device direct access solution requires pinning memory, which conflicts with the dynamic memory movement solution adopted by TwinVisor.

[0133] On the other hand, the main functions of TwinVisor are implemented in N-visor. N-visor runs in NormalWorld and cannot access security devices, so it cannot complete the work of configuring device passthrough for confidential virtual machines.

[0134] The ARM CCA platform itself has not yet been widely used and deployed, and is not suitable as a mainstream solution for major cloud vendors at this stage. In addition, ARM CCA itself does not have the ability to directly connect confidential virtual machines to secure devices.

[0135] Therefore, the confidential virtualization solution for the ARM platform will face the following technical problems when performing secure device pass-through:

[0136] 1) The number of secure world memory regions that can be used by confidential virtual machines is limited, which limits the secure memory that confidential virtual machines on the host can access, further limiting the number of confidential virtual machines.

[0137] 2) If Figure 1 As shown in the figure, the N-visor in the normal world can manage resources for normal virtual machines and confidential virtual machines. If the device management function code is added to the N-visor to implement direct access to virtual machines, since the N-visor belongs to the normal world, it cannot access the secure devices in the secure world, and thus cannot implement direct access to the secure devices of the confidential virtual machines (S-VMs) in the secure world.

[0138] 3) Although Figure 1 The S-visor shown is in the secure world and can access secure devices. However, the S-visor runs at the S-EL2 privilege level. As the TCB of the entire system, the S-visor isolates the N-visor and the S-VM. The TCB must not be too complex and must be kept simple. Therefore, the device management function code cannot be added to the S-visor.

[0139] Then for the ARM platform, in order to achieve direct access of the confidential virtual machine in the secure world to the secure device, this application provides a device direct access solution to solve the above three technical problems. The following uses TwinVisor as an example to support secure device direct access to illustrate the technical solution of this application. When the technical solution of this application is applied to other system architectures of the ARM platform (such as ARM CCA, etc.), the implementation principle is similar and will not be repeated here.

[0140] Before introducing the technical solution of the present application, the two-level address translation mechanism based on the page table adopted by the ARM platform is introduced below to facilitate understanding of the technical solution of the present application.

[0141] The first stage page table is a mapping relationship from the guest virtual address (GVA) to the intermediate physical address (IPA). The second stage page table can be a mapping relationship from IPA to the host physical address (HPA). The first stage translation is performed by the virtual machine (both the virtual machine in the secure world and the virtual machine in the normal world), and the virtual machine can translate GVA into IPA through the first stage page table. The second stage translation is managed and controlled by the hypervisor, such as the Figure 3 The S-visor in the IO can translate IPA into HPA through the second-stage page table. The translation of these two stages is independent of each other and does not interfere with each other.

[0142] In addition, the first-stage page table can be classified into a first-stage page table for CPU access and a first-stage page table for security device access (such as IO device). The second-stage page table can be classified into a second-stage page table for CPU access and a second-stage page table for security device access (such as IO device). The access from the virtual address to the physical address can be realized by two translations of the first-stage page table and the second-stage page table.

[0143] Among them, the memory that can only be accessed by the secure world can be called secure memory, and the memory accessible to ordinary virtual machines can be called ordinary memory (or non-secure memory).

[0144] Figure 3 The following is a schematic diagram of the system architecture of the present application that supports secure device pass-through based on TwinVisor, which is shown as an example.

[0145] like Figure 3As shown, the system may include a part in the normal world and a part in the secure world. The part in the normal world may include normal virtual machines 1 to n running at privilege levels N-EL0 and N-EL1, where n is a positive integer. The normal virtual machines may also be represented by N-VM, and the Hypervisor (represented by N-visor) in the normal world running at the N-EL2 privilege level.

[0146] Among them, N-visor can be used for ordinary virtual machines 1 to ordinary virtual machines n, and Figure 3 The confidential virtual machines 1 to n shown in the figure perform resource management. For example, N-visor can be used to perform most virtual machine fallback processing, such as second-stage page table fault exception processing, MMIO processing, etc.

[0147] The part in the secure world may include confidential virtual machines 1 to confidential virtual machines n running at privilege levels S-EL0 and S-EL1, where n is a positive integer, and confidential virtual machines (also called secure virtual machines) may also be represented by S-VM. A secure virtual machine is a virtual machine running in a trusted execution environment (TEE).

[0148] In addition, compared with the existing technologies (such as Figure 1 ), the secure world part of the present application may also include service virtual machines running at privilege levels S-EL0 and S-EL1.

[0149] Among them, the service virtual machine (also known as SSrv-VM) is a special confidential virtual machine. Although it is a virtual machine running in the secure world, it is not used to run customer applications, but to manage security devices (such as Figure 3 As shown, IO devices 1 to IO devices m), as well as direct connection of security devices to S-VM, etc., are configured to implement configuration management of direct connection of security devices to confidential virtual machines, but the service virtual machine does not belong to TCB.

[0150] For example, a Hypervisor (here referred to as Tri-visor) may run in the special confidential virtual machine, but the Hypervisor is mainly used for configuration management of secure device direct access to confidential virtual machines (such as the above-mentioned confidential virtual machine 1 to confidential virtual machine n).

[0151] The secure world portion may also include a Hypervisor (represented by S-visor) located in the secure world and running at the S-EL2 privilege level.

[0152] The S-visor runs at the S-EL2 privilege level of the secure world. It serves as the TCB of the entire system and is responsible for security protection. This application improves the S-visor (described in detail below) so that the S-visor can control S-VMs (such as Figure 3 The SSrv-VM is isolated from any one of the confidential virtual machines 1 to n shown in the figure), N-visor and Tri-Visor. This prevents untrusted N-visor, S-VM and Tri-Visor from attacking each other. Since the SSrv-VM is not part of the TCB, the S-visor needs to ensure the isolation between the SSrv-VM and the S-VM.

[0153] It should be understood that the number of common virtual machines and the number of confidential virtual machines may be the same or different, and there is no limitation here.

[0154] like Figure 3 As shown, the N-visor and the S-visor can be connected through trusted firmware.

[0155] like Figure 3 As shown, the hardware layer may include memory devices, such as dynamic random access memory (DRAM), etc., which is not limited here.

[0156] The hardware layer may also include IO devices 1 to IO devices m, where m is an integer greater than or equal to 1. The IO devices may be network-supported or storage-supported devices, etc., which are not limited here. The IO devices here are also referred to as PCIe devices or security devices hereinafter. The IO devices 1 to IO devices m here represent security devices for passing directly to the S-VM.

[0157] In the system of the present application, in order to achieve direct access to the security devices of the S-VM, considering that the management function of the security devices is added in the N-visor, the N-visor belongs to the ordinary world and cannot access the security devices in the secure world, so the direct access to the security devices of the S-VM cannot be achieved. If the management function of the security devices is added in the S-visor, considering that the S-visor is a TCB, in order to ensure the simplicity of the TCB, the management function of the security devices cannot be added in the S-visor.

[0158] To this end, the system of the present application implements a secure device management method based on the above-mentioned SSrv-VM. The present application writes the device management module into a special confidential virtual machine (also called a service virtual machine, SSrv-VM, also expressed as a second secure virtual machine). And SSrv-VM runs at the S-EL0 and S-EL1 privilege levels, so that SSrv-VM can be managed by S-visor (which may include the communication module of the present application) under the condition of ensuring the streamlining of TCB. In addition, a complete operating system (such as Linux) can be run in the service virtual machine to run programs for secure device management and direct access to S-VM devices (such as the above-mentioned Tri-visor). In this way, on the one hand, TCB will not be expanded, and on the other hand, the function of device management can be provided by reusing a complete Linux to reduce the complexity of implementation. This design can not only simplify the workload in the development stage, but more importantly, it can rely on the active Linux community to simplify subsequent iterations and maintenance work. In addition, SSrv-VM can be used to manage and configure devices, but does not participate in the management of the data plane, thereby avoiding the additional sinking and forwarding overhead.

[0159] This application ensures the security and isolation of confidential virtual machine execution by running a minimized TCB module (S-visor) in Secure World. And this solution is based on ARM TrustZone, a hardware extension widely available on existing ARM servers. Compared with ARM CCA, the large number of adaptable servers makes this application easier to be accepted and applied by the majority of cloud service providers.

[0160] In addition, the security device management technology based on the service virtual machine can reuse the full-featured OS management security device pass-through of the virtual machine in the secure world through a software solution. By running Tri-visor in the confidential virtual machine, on the one hand, it prevents the device management function from being written to S-visor, causing the TCB to be too large and the redundancy of repeated development; on the other hand, it solves the problem that N-visor cannot access the security device, so that the pass-through function of the security device can be enabled for the confidential virtual machine in the Secure World. Because the device management function reuses Linux, relying on the powerful Linux community and complete ecology, cloud vendors do not need to maintain developers specifically for this part of the function, but only need to cooperate with the mainline Linux update.

[0161] In the embodiments of the present application, Figure 3As shown, the N-visor, S-visor, and SSrv-VM do not trust each other. The N-visor can control the S-visor to perform corresponding operations to achieve resource management of the confidential virtual machine, etc. The service virtual machine can control the S-visor to perform corresponding operations to achieve device pass-through configuration of the confidential virtual machine, etc. Among them, the S-visor can perform corresponding operations according to the instructions of the N-visor or the service virtual machine, and perform data checks when performing corresponding operations to achieve access control and ensure isolation between the N-visor, S-visor, and SSrv-VM.

[0162] The system of the present application is described below in combination with different examples.

[0163] Example 1: Register a security device to a service virtual machine.

[0164] Combine the following Figure 4a To describe the system initialization process of this application.

[0165] like Figure 4a As shown, the process may include at least the following steps:

[0166] S101, N-visor initializes the security device.

[0167] Among them, the safety device is as follows Figure 4a The IO devices 1 to IO devices m are shown.

[0168] Specifically, during the host startup phase, N-visor can traverse the device tree and scan the devices one by one. In this phase, in order to facilitate the ordinary world to access the secure device, PCIe devices (such as Figure 4a The device types of the IO devices 1 to IO devices m are set as common devices. N-visor accesses the configuration space of the PCIe device through memory mapping I / O (MMIO) to configure the PCIe device.

[0169] Among them, N-visor can write all PCI-BARs to 1, and then read the value of PCI-BAR (an example of the initialization result of PCIe device) to determine the address space size and type corresponding to each PCI-BAR. N-visor determines the granularity of device MMIO by reading the position of the lowest bit 1. For example, if the lowest bit 1 is the 12th bit, the granularity of MMIO is 4K. N-visor writes the starting address of the address space to the high bits of BAR. Thereafter, when the physical address (PA) falls in the address space pointed to by the PCI-BAR, the root complex can forward the address request to the PCIe device.

[0170] This application does not impose any restrictions on the initialization process of the security device, and it can be processed according to the process of the prior art.

[0171] In this way, the N-visor may obtain the initialization result of each security device. For example, the initialization result may include the HPA (eg, MMIO address range) of the security device.

[0172] In this way, IO devices 1 to IO devices m are managed by N-visor, and after the service virtual machine is initialized, N-visor can transfer the management rights of these IO devices to the service virtual machine. In the process of transferring the management rights of IO devices to the service virtual machine, N-visor can notify S-visor which IO devices to transfer, and S-visor can map the physical address range of these IO devices to the service virtual machine (for example, mapped to the IPA corresponding to the service virtual machine). The following S102 describes an implementation method of mapping the physical address range of IO devices to the service virtual machine in the form of a second-stage page table. However, the implementation method of mapping the physical address range of these IO devices to the service virtual machine is not limited to the page table method.

[0173] S102, the N-visor may control the S-visor to configure a second-stage page table belonging to the service virtual machine based on the initialization result of the security device.

[0174] The N-visor may send the initialization result of the security device to the S-visor. For example, the initialization result includes the MMIO address range (eg, the value of the PCI-BAR) of each security device that needs to be managed by the service virtual machine.

[0175] Optionally, the S-visor can check the MMIO address range of the security device to prevent different S-VMs from sharing the same security device, ensuring data isolation between different confidential virtual machines.

[0176] The S-visor can record the MMIO address range of the security devices that have passed the inspection in its own secure memory. However, the S-visor itself cannot identify which security device each MMIO address belongs to, nor can it identify which IPA of the corresponding security device the MMIO address is mapped to. In this way, during the system initialization phase, the S-visor can save the physical address range of the security devices managed by the service virtual machine and map the physical address range to the service virtual machine (which can also be described as associating the physical address range with the service virtual machine), so that during the system initialization phase, all security devices are directly connected to the service virtual machine for registration.

[0177] In a possible implementation manner, the S-visor may record the MMIO address range of the security device managed by the service virtual machine by configuring a second-stage page table belonging to the service virtual machine.

[0178] Specifically: N-visor can configure the page table content of the second-stage page table for the service virtual machine based on the initialization result (for example, the MMIO address range of the security device), and control S-visor to configure the second-stage page table for the service virtual machine according to the page table content, and S-visor can make the second-stage page table effective to the register of the CPU (the specific method can refer to the existing technology and is not limited here), so that the value of the register can point to the second-stage page table.

[0179] The second-stage page table belonging to the service virtual machine can indicate the physical address range of the security device managed by the service virtual machine.

[0180] For example, the second stage page table may be capable of indicating a security device (eg Figure 3 The page table of the mapping relationship between the IPA (or other device identification information) and MMIO of IO devices 1 to IO devices m) shown in the figure does not limit what identification information of the IO device is mapped to the MMIO address in the page table.

[0181] In this way, the S-visor can record the physical address range of the security device managed by the service virtual machine during the system initialization phase (for example, through the second-stage page table method, or other methods, not limited), so as to directly pass the security device to the service virtual machine, so that the service virtual machine can manage the security device. Figure 3 The IPA information of IO devices 1 to IO devices m shown (it may also be other identification information that can indicate the IO devices, which is not limited here), wherein the service virtual machine can obtain the IPA information of the above-mentioned IO devices 1 to IO devices m from N-visor through S-visor during the system initialization phase.

[0182] Among them, S-visor can configure the second stage page table of the direct-pass security device (here IO device 1 to IO device m) for the service virtual machine under the control of N-visor. Among them, the second stage page table belonging to the service virtual machine here can indicate the physical address range that the service virtual machine can access to IO device 1 to IO device m, so that the physical address range of IO device 1 to IO device m can be mapped to the service virtual machine, realizing the direct pass of the service virtual machine to IO device 1 to IO device m.

[0183] Of course, the implementation method of S-visor mapping the physical address range of IO device 1 to IO device m to the service virtual machine is not limited to the method of configuring the second-stage page table belonging to the service virtual machine described in S102, wherein the second-stage page table can map the physical address range of IO device 1 to IO device m, and can also be other implementation methods, which are not limited here.

[0184] In addition, before the S-visor configures the second-stage page table for the service virtual machine, the S-visor can check the page table content of the second-stage page table (the specific checking scheme can refer to the existing technology and is not limited here) to prevent different S-VMs from sharing the same security device and ensure data isolation between different confidential virtual machines. After the S-visor passes the page table content check of the second-stage page table, the S-visor can configure the security devices it manages (such as Figure 4a The second stage page table of IO devices 1 to IO devices m) shown in FIG. The second stage page table may map the physical address range of the security device managed by SSrv-VM.

[0185] In addition, when the S-visor configures the second-stage page table of the security device it manages for the service virtual machine, the N-visor controls how the S-visor is configured and the content of the configured page table. The S-visor only writes the configured content into the second-stage page table belonging to the service virtual machine and saves the second-stage page table to the secure memory. In this way, during the system initialization phase, the S-visor can directly pass all security devices to the service virtual machine for registration under the control of the N-visor.

[0186] Optionally, during the system initialization phase, the service virtual machine may also perform a security check on the security devices it manages (e.g. Figure 4a The IO devices 1 to IO devices m) shown in the figure create a first-stage page table, which can indicate the mapping relationship between the GVA and IPA of the above-mentioned security devices managed by the service virtual machine. In this way, the service virtual machine can obtain the specific content of the IPA of each security device it manages.

[0187] For example, N-visor controls S-visor to notify the service virtual machine that a certain IPA belongs to IO device 1. Then, the service virtual machine can establish the above-mentioned first-stage page table according to the IPA of the IO device 1. The first-stage page table can be mapped to the IPA of the IO device 1. There is no restriction on which virtual address is mapped to the IPA.

[0188] Optionally, during the system initialization phase, the S-VM may also establish a first-stage page table of its accessible addresses (memory addresses, security device addresses). The specific process will not be repeated here, and the process of any S-VM in the prior art establishing a first-stage page table for its accessible address resources may be referred to.

[0189] Through the above S101 and S102, all PCIe devices installed on the host can be accessed and managed by SSrv-VM, so TZPC (a kind of hardware) can dynamically connect PCIe devices (such as Figure 4a The device types of the IO devices 1 to IO devices m shown are readjusted to safety devices, and thereafter the N-visor will no longer have the authority to directly access the IO devices 1 to IO devices m.

[0190] Although the secure world can access the memory of the normal world, since the Tri-visor runs in the SSrv-VM, the N-visor, S-visor, and SSrv-VM do not trust each other, so the Tri-visor cannot access the data structure maintained in the N-visor. In order to enable the SSrv-VM to manage the security device and achieve direct access to the security device of the S-VM without making the program in the SSrv-VM too complicated, the system of the present application can directly use the initialization result of the security device (such as the HPA of the security device) of the N-visor instead of initializing the security device. To this end, in this embodiment, the N-visor can control the S-visor to configure the above-mentioned second-stage page table belonging to the SSrv-VM when the SSrv-VM is initialized. The second-stage page table can be mapped to the HPA of the security device managed by the SSrv-VM to achieve shared access to the data structure of the N-visor (such as the HPA of the security device).

[0191] In this way, N-visor allocates the HPA of the security device to the service virtual machine, and after S-visor checks and passes the allocated HPA, S-visor can, under the control of N-visor, add a second-stage page table that can be mapped to the HPA of the security device to the service virtual machine, thereby directly connecting the security device to the service virtual machine.

[0192] Example 2: The service VM configures the pass-through security device.

[0193] go through Figure 4a process, such as Figure 4b As shown by the bidirectional thick arrows, IO device 1 has been directly connected to the service virtual machine. Of course, IO devices 2 to IO devices m are also directly connected to the service virtual machine.

[0194] Then the service virtual machine can configure the direct security device. Figure 4b The schematic diagram exemplarily shows the process of configuring the security device by the service virtual machine.

[0195] like Figure 4b As shown, the process may include the following steps:

[0196] S201, the service virtual machine sends a device configuration request to the S-visor.

[0197] For example, when configuring a security device that has been registered to the service virtual machine (for example Figure 4a When the IO device 1 is shown, the service virtual machine can configure PCI-bar and SMMU for the IO device 1.

[0198] Among them, PCI-bar is the second stage page table for CPU access, and SMMU is the second stage page table for IO device 1 access.

[0199] The device configuration request in S201 may be a request to configure a PCI-bar for the IO device 1 or a request to configure an SMMU for the IO device 1 .

[0200] S202: S-visor configures IO device 1.

[0201] Optionally, during the process of configuring the security device of the service virtual machine, the S-visor may check the configuration request of the service virtual machine to the security device to ensure data isolation between the security devices accessed by the N-visor, S-VM and the service virtual machine.

[0202] For example, when the service virtual machine requests to perform sensitive operations such as modifying the SMMU page table of IO device 1, the S-visor can check whether IO device 1 is a secure device that the service virtual machine has directly passed through through the second-stage page table belonging to SSrv-VM configured in the above example 1. If so, the S-visor can modify the SMMU page table of IO device 1 under the control of the service virtual machine. If not, the S-visor can reject the corresponding configuration operation of the service virtual machine.

[0203] In an embodiment of the present application, through Example 1, the S-visor adds a second-stage page table about security devices (such as IO devices 1 to IO devices m) to the service virtual machine, so that these security devices are directly passed to the service virtual machine. Then, the service virtual machine can access and configure the PCI-bar and SMMU page tables of these security devices. Among them, when the service virtual machine configures certain sensitive operations (such as SMMU configuration) on the directly passed security device, the S-visor can check the sensitive operation through the above-mentioned second-stage page table belonging to the service virtual machine that has been added in Example 1, to ensure that the service virtual machine only performs configuration operations on the security devices that are directly passed to the service virtual machine, and cannot perform configuration operations on security devices that are not directly passed.

[0204] In a possible implementation, Figure 4b As shown by the bidirectional thick arrow, IO device 1 has been directly connected to the service virtual machine. Then, after the service virtual machine performs configuration operations for the security device that is directly connected, the service virtual machine can directly access IO device 1.

[0205] Specifically, for example, the service virtual machine can translate the GVA of the IO device 1 to be accessed into an IPA (e.g., the IPA of IO device 1) through the first-stage page table created by the service virtual machine. The CPU uses the IPA to check the second-stage page table belonging to the service virtual machine (generated by S102 in the above example 1), and the CPU can determine the HPA mapped by the IPA in the second-stage page table, thereby accessing the HPA to achieve direct access of the service virtual machine to IO device 1.

[0206] In this way, the service virtual machine can access the HPA of the security device through the GVA to achieve direct access to the security device without copying the access content between the non-secure zone and the secure zone.

[0207] After all security devices on the host are registered with the service VM, the S-visor can transfer the security devices registered (passed through) to the service VM to the S-VM under the control of the service VM, so that the S-VM can directly access the security devices. Figure 4a In the case where the IO device 1) shown is not directly connected to the S-VM, the service virtual machine directly accesses the security device. In the case where a security device has been transferred to the S-VM, the service virtual machine no longer has the authority to access the security device. The S-visor can be used to implement access control of the security device to prevent the security device directly connected to the confidential virtual machine from being accessed by the service virtual machine, other confidential virtual machines or ordinary virtual machines.

[0208] The following describes the process of transferring the security device that the service virtual machine directly connects to the S-VM in combination with Example 3, so as to connect the security device directly to the S-VM.

[0209] Example 3: The service virtual machine directly passes the security device to the S-VM (the same applies to canceling the direct pass-through of the security device to the S-VM).

[0210] Figure 4c The process of the system of the present application directly connecting the confidential virtual machine 1 to the security device (here, IO device 1) is exemplified.

[0211] like Figure 4c As shown, the process may include the following steps:

[0212] S302a, the S-visor receives a first device direct connection request of a confidential virtual machine 1 (hereinafter referred to as S-VM1).

[0213] The first device direct connection request may include information indicating the security device (eg, IO device 1) to which S-VM1 requests direct connection. The information indicating the security device to be direct connected may be the identifier or MMIO address information of the security device, which is not limited here.

[0214] In this embodiment, the S-visor may receive a device direct connection request for performing device direct connection on the S-VM1 from the S-VM1. In other embodiments (such as the embodiment of S302b), the S-visor may receive a device direct connection request for performing device direct connection on the S-VM1 from the N-visor. The principles are similar and will not be repeated here.

[0215] Referring to Example 1, the IO device 1 is currently passed directly to the service virtual machine, so that the IO device 1 is managed by the service virtual machine. Then the service virtual machine needs to configure the IO device 1 to S-VM1 so that S-VM1 can manage the IO device 1 instead of the service virtual machine.

[0216] In other embodiments, the above S302a may also be replaced by Figure 4c S302b is indicated by the dashed arrow shown.

[0217] S302b, S-visor may receive a second device direct pass request from N-visor, and execute the following S303 and S305 in response to the second device direct pass request, wherein the service virtual machine still executes S304 after S303 to direct pass IO device 1 to S-VM1. The second device direct pass request may include information indicating that S-VM1 needs to direct pass IO device 1.

[0218] That is to say, S-visor can request the service virtual machine under the control of the confidential virtual machine (here S-VM1) that needs a pass-through device or N-visor to pass through at least one security device managed by the service virtual machine to the confidential virtual machine (for example, S-VM1). The principles of the system implementation process in the two scenarios are similar and will not be repeated here.

[0219] It should be understood that when the confidential virtual machine controls the S-visor to perform any operation, it also sends a request to the N-visor through the ordinary virtual machine in the secure world, and the N-visor then transfers the request to the confidential virtual machine through the S-visor, so that the confidential virtual machine can control the S-visor to perform the corresponding operation (for example, the S-visor requests the IPA of the security device to be passed through to the service virtual machine through S303).

[0220] Similarly, when N-visor executes the above S302b to control S-visor to request the service virtual machine to perform device pass-through on a confidential virtual machine, N-visor also responds to the request of the corresponding ordinary virtual machine, thereby triggering the execution of S302b, rather than N-visor actively sending a second device pass-through request to S-visor.

[0221] In one example, combined with the scenario of S302b above, N-visor may create a new instance of S-VM1 related to a common virtual machine (e.g., common virtual machine 1) based on a request of the common virtual machine. During the process of N-visor creating an instance of S-VM1, N-visor may send a second device direct connection request to S-visor through S302b to request direct connection of a certain device to S-VM1.

[0222] S303: The S-visor may send a third device direct communication request to the service virtual machine in response to the first device direct communication request.

[0223] Among them, the third device direct request may include instructing the service virtual machine to send information about the IPA of IO device 1 (for example, identified by the ID of IO device 1) to request the service virtual machine for the IPA information of the IO device it manages.

[0224] Among them, through example 1, the service virtual machine saves the IPA information of each IO device among the IO devices 1 to IO devices m managed by it.

[0225] Optionally, the S-visor may check whether the IO device 1 has been directly connected to other confidential virtual machines in response to the first device direct connection request. If the S-visor determines that the IO device 1 has been directly connected to other confidential virtual machines, the S-visor may reject the first device direct connection request of S-VM1.

[0226] Optionally, when the S-visor determines that the IO device 1 is not directly connected to other confidential virtual machines, the S-visor may send a third device direct connection request to the service virtual machine, for example, to request the IPA of the IO device 1 .

[0227] When responding to a device pass-through request from S-VM1 or N-visor, S-visor may mark the security device to be passed through to indicate that the security device has been assigned to the current confidential virtual machine (e.g., S-VM1). Before marking IO device 1, S-visor may check the marking information of IO device 1 to determine whether IO device 1 has been passed through to other confidential virtual machines.

[0228] The confidential virtual machine of the present application exclusively uses the security device. In other words, the service virtual machine can only pass a security device directly to one confidential virtual machine, but not to multiple confidential virtual machines, thereby preventing the sharing of IO devices accessed by multiple confidential virtual machines, thereby preventing Dos attacks on virtual devices.

[0229] S304: The service virtual machine may send control information indicating configuration of IO device 1 for S-VM1 in response to the third device direct connection request.

[0230] For example, the service virtual machine can respond to the third device direct pass request and send the IPA information of the IO device 1 it manages to the S-visor through the control information. For example, the first-stage page table of the service virtual machine translates from GVA to IPA to obtain the IPA information of the IO device 1. There is no limitation here.

[0231] Optionally, the control information in S304 does not need to include information indicating S-VM1, because, when the S-visor forwards the device ID information through the third device direct connection request, it already knows which S-VM (here is S-VM1) is being connected to, so the above control information of the service virtual machine in response to the third device direct connection request must correspond to S-VM1.

[0232] S305 , the S-visor may configure a second-stage page table belonging to S-VM1 in response to the control information sent by the service virtual machine.

[0233] Among them, S-visor can configure the second stage page table of the direct-pass security device (here, IO device 1) for S-VM1 under the control of the service virtual machine. Among them, the second stage page table belonging to S-VM1 here can indicate the physical address range that S-VM1 can access to IO device 1, so that the physical address range of IO device 1 can be mapped to S-VM1, and S-VM1 can directly pass through IO device 1.

[0234] Of course, the implementation method of S-visor mapping the physical address range of IO device 1 to S-VM1 is not limited to the configuration method of the second-stage page table belonging to S-VM1 described in S305, wherein the second-stage page table can map the physical address range of IO device 1, and can also be other implementation methods, which are not limited here.

[0235] In this way, under the control of the service virtual machine, the S-visor can record the physical address range of the security devices accessible to S-VM1, for example, corresponding to the second-stage page table for IO device 1 mentioned above, but of course it is not limited to the page table method.

[0236] Optionally, when the S-visor configures the second-stage page table about IO device 1 for S-VM1 in response to the control information sent by the service virtual machine, the S-visor may use the IPA information sent by the service virtual machine to query the second-stage page table belonging to the service virtual machine (for example, the second-stage page table described in S102 in Example 1) to determine whether the IPA information is mapped with MMIO information in the second-stage page table of the security device managed by the service virtual machine, thereby determining whether the security device (here is IO device 1) indicated by the service virtual machine to be passed directly to S-VM1 is the security device managed by the service virtual machine. In other words, the S-visor can check whether the security device that the service virtual machine currently wants to pass directly to S-VM1 is a security device within the management authority of the service virtual machine.

[0237] In one possible implementation, when the S-visor determines that the IPA information indicated by the control information is not mapped with MMIO information in the second-stage page table of the security device managed by the service virtual machine (for example, the second-stage page table described in S102 in Example 1), it indicates that the IO device 1 to be passed directly to S-VM1 has been passed directly to other S-VMs. The S-visor may refuse to pass the IO device 1 directly to S-VM1 in response to the control information to ensure that the IO devices accessed by different confidential virtual machines are different.

[0238] In one possible implementation, when the S-visor determines that the IPA information indicated by the control information is mapped with MMIO information in the second-stage page table of the security device managed by the service virtual machine (for example, the second-stage page table described in S102 in Example 1), it indicates that the IO device 1 to be directly passed to S-VM1 is a security device managed by the service virtual machine, and the security device has not been directly passed to other S-VMs. Then, the S-visor can configure the second-stage page table of the security device directly passed to S-VM1 in response to the control information, so as to directly pass the IO device 1 to S-VM1.

[0239] Among them, when the S-visor configures the second-stage page table of IO device 1 belonging to S-VM1 under the control of the service virtual machine, the S-visor can query the MMIO information that can be mapped to the IPA issued by the service virtual machine in the second-stage page table of the security device managed by the service virtual machine (for example, the second-stage page table described in S102 in Example 1), thereby establishing a mapping relationship between the IPA issued by the service virtual machine and the MMIO information mapped thereto queried by the S-visor, and constructing the second-stage page table of the security device (IO device 1) belonging to the S-VM1.

[0240] For example, the second-stage page table may be a page table that can indicate the mapping relationship between the IPA and MMIO of the IO device 1. The specific mapping from which virtual address to MMIO address in the page table is mapped is not limited here.

[0241] In addition, under the control of the service virtual machine, the S-visor can also make the second-stage page table configured for S-VM1 about IO device 1 effective to the CPU register (the specific method can refer to the existing technology and is not limited here), so that the value of the register can point to the second-stage page table.

[0242] In other embodiments, the S-visor may also configure the second-stage page table of the memory device belonging to S-VM1 under the control of the N-visor to limit the memory address range of the secure memory accessible to S-VM1, wherein the second-stage page table may map the memory device accessible to S-VM1 ( Figure 4c The physical address range in the memory device shown in FIG. 1 . And the S-visor can, under the control of the N-visor, validate the second-stage page table to the register of the CPU (the specific method can refer to the prior art and is not limited here), so that the value of the register can point to the second-stage page table.

[0243] So if Figure 4c As shown by the dotted double-bold arrow, IO device 1 was originally a secure device directly connected to the service virtual machine. However, the solid double-bold arrow currently pointing to the IO device 1 has pointed to S-VM1, indicating that the IO device 1 has switched from the service virtual machine to being directly connected to S-VM1. In this way, S-VM1 can directly access IO device 1 through the solid double-bold arrow without the need for the CPU to copy the accessed data packets between the secure memory and the non-secure memory, thereby reducing the performance overhead of the confidential virtual machine accessing the IO device.

[0244] In a possible implementation, the second-stage page table about IO device 1 used by S-visor is originally the data about IO device 1 in the second-stage page table belonging to the service virtual machine (for example, the second-stage page table obtained by S102 of Example 1). If the current IO device 1 has been directly passed to S-VM1, then S-visor can switch the second-stage page table about IO device 1 from the second-stage page table belonging to the service virtual machine to the second-stage page table about IO device 1 belonging to S-VM1, so as to directly pass IO device 1 to S-VM1. Optionally, S-visor can also delete the mapping content of IO device 1 that has been directly passed to S-VM1 in the second-stage page table belonging to the service virtual machine that it saves.

[0245] In addition, S-visor is connected to S-VM1 through the security device (here Figure 4c When the IO device 1 shown in the figure is connected, the service virtual machine controls how the S-visor configures the second-stage page table belonging to the S-VM1 regarding the direct-access security device, as well as the configured page table content. The S-visor simply writes the configured content into the second-stage page table belonging to the S-VM1, and saves the second-stage page table to the secure memory. In this way, the S-visor can, under the control of the service virtual machine, direct-access all the security devices possessed by the S-VM1 to the S-VM1, so that the S-VM1 can directly access the security devices without copying the accessed data between the non-secure memory and the secure memory.

[0246] The implementation principle of the similar steps in Example 3 is similar to that of Example 1, except that Example 1 describes the process of the service virtual machine directly connecting to the security device, while Example 3 describes the process of the service virtual machine directly connecting to the security device for the confidential virtual machine 1.

[0247] Example 4: A confidential virtual machine configures a pass-through security device.

[0248] The implementation principle of Example 4 is similar to that of Example 2, except that Example 2 describes configuring a security device that is directly connected to the service virtual machine for the service virtual machine, while Example 4 describes configuring a security device that is directly connected to the confidential virtual machine 1 for the confidential virtual machine 1.

[0249] go through Figure 4c process, such as Figure 4d As shown by the thick double solid arrow, IO device 1 has been directly connected to confidential virtual machine 1. Then confidential virtual machine 1 can configure the security device that is directly connected. Figure 4d The schematic diagram of the process of configuring a security device by a confidential virtual machine is exemplified.

[0250] like Figure 4dAs shown, the process may include the following steps:

[0251] S401, confidential virtual machine 1 sends a device configuration request to S-visor.

[0252] For example, a security device (such as Figure 4d When the IO device 1 is shown, the confidential virtual machine 1 can configure PCI-bar and SMMU for the IO device 1 and other configuration operations.

[0253] Among them, PCI-bar is the second stage page table for CPU access, and SMMU is the second stage page table for device (here, IO device 1) access.

[0254] The device configuration request in S401 may be a request to configure a PCI-bar for IO device 1 or a request to configure an SMMU for IO device 1, etc., which is not limited here.

[0255] S402: S-visor configures IO device 1.

[0256] Optionally, during the process of confidential virtual machine 1 configuring IO device 1, S-visor may check the configuration request of confidential virtual machine 1 to the security device to ensure data isolation between the security devices accessed by N-visor, S-VM and service virtual machine.

[0257] For example, when confidential virtual machine 1 requests to perform sensitive operations such as modifying the SMMU page table of IO device 1, the S-visor can check whether IO device 1 is a security device that confidential virtual machine 1 has directly connected to through the second-stage page table belonging to S-VM1 configured in the above example 3 (for example, the second-stage page table obtained in the above S305); if so, the S-visor can modify the SMMU page table of IO device 1 under the control of confidential virtual machine 1; if not, the S-visor can reject the corresponding configuration operation of confidential virtual machine 1.

[0258] For example, S-visor may query the second-stage page table belonging to S-VM1 (such as the second-stage page table obtained in S305 above). If no mapping about IO device 1 is found, it means that S-VM1 does not have access rights to IO device 1, thereby rejecting the device configuration request of S-VM1.

[0259] For another example, when S-visor queries the second-stage page table belonging to S-VM1 (such as the second-stage page table obtained in S305 above), when it finds the mapping of IO device 1, it means that S-VM1 has access rights to IO device 1, and thus calls and forwards the device configuration request to implement the configuration of IO device 1.

[0260] In an embodiment of the present application, through Example 3, the S-visor adds a second-stage page table about the security device (here, IO device 1) to the confidential virtual machine 1, so that the IO device 1 is directly passed to the confidential virtual machine 1. Then, the confidential virtual machine 1 can access and configure the PCI-bar and SMMU page tables of the IO device 1. Among them, when the confidential virtual machine 1 configures certain sensitive operations (such as SMMU configuration) on the directly passed security device, the S-visor can check the sensitive operation through the above-mentioned second-stage page table belonging to the confidential virtual machine 1 that has been added in Example 3, to ensure that the confidential virtual machine 1 only performs configuration operations on the security device that is directly passed to the confidential virtual machine 1, and cannot perform configuration operations on security devices that are not directly passed.

[0261] Combine the following Figure 4d To illustrate the process of S-VM1 configuring the second stage page table (such as SMMU) for the pass-through security device (such as the above IO device 1) for access by the security device (such as the above IO device 1):

[0262] 1. The user-mode hypervisor of Normal World allocates an IPA from the guest address space for the secure device DMA to use, but since it runs in Normal World, this step does not actually create DMA memory.

[0263] 2. Normal World user-mode hypervisor requests N-visor to write the SMMU secondary page table, which is then forwarded to SSrv-VM via S-visor. A secure memory for DMA is created in SSrv-VM and attempts to rewrite the SMMU secondary page table.

[0264] 3. SSrv-VM's access to the SMMU secondary page table triggers a second-stage page table page fault exception. After sinking to the S-visor, the S-visor needs to check the sinking reason and the exception address to determine whether the current page fault exception is caused by SSrv-VM's attempt to rewrite the SMMU second-stage page table. If so, the normal page fault processing logic is skipped, and the S-visor directly helps SSrv-VM complete the task of rewriting the page table, thereby configuring the SMMU belonging to S-VM1, and then updating the PC register in SSrv-VM so that S-VM1 does not trigger the exception again.

[0265] Combine the following Figure 4d To illustrate the process of S-VM1 configuring the second-stage page table (such as PCI-bar) for CPU access to the direct security device (such as the above IO device 1):

[0266] 1. When starting a confidential virtual machine, the user needs to pass the PCIe address of the pass-through security device (such as IO device 1) that the user needs to access to the Normal World user-mode virtual machine hypervisor. The Normal World user-mode virtual machine hypervisor requests the size and properties of the device memory from the N-visor based on the PCIe address. Although the N-visor can no longer access the security device at this time, it can request relevant data from the S-visor and pass it to the N-visor.

[0267] For example, the S-visor can obtain the identification information (e.g., MMIO address) of the security device (e.g., IO device 1) that is directly passed to S-VM1 from the N-visor, because, during the system initialization phase of Example 1, the N-visor has saved the identification information of the security device by initializing the security device.

[0268] 2. The Normal World user-mode hypervisor can map the HPA (e.g., MMIO address) of the security device that needs to be directly connected to S-VM1 to the Guest address space to obtain the HVA, and allocate the IPA from the Guest address space according to the size of the device memory to complete the binding of the HVA to the IPA. Finally, the Normal World user-mode hypervisor fills the IPA into the virtual PCI-BAR.

[0269] 3. The Normal World user-mode hypervisor can synchronize the address mapping relationship of HVA->IPA to the N-visor and S-visor.

[0270] 4. When S-VM1 first attempts to access the address in the virtual PCI-BAR of the IO device 1 to access the device memory, the second-stage page table belonging to the S-VM1 in the S-visor for CPU access has not been mapped, thus triggering a second-stage page table page fault exception. After checking the IPA where the error occurred, the S-visor forwards the N-visor to execute the processing logic of the second-stage page table page fault exception. Any modifications made to the page table by the N-visor will not actually take effect. The S-visor will check the modifications made by the N-visor to the second-stage page table belonging to the S-VM1 for CPU access. After confirming that there is no error, the S-visor will synchronize the modifications to the second-stage page table to the real second-stage page table belonging to the S-VM1 for CPU access.

[0271] In a possible implementation, Figure 4d As shown by the bidirectional thick arrows, IO device 1 has been directly connected to confidential virtual machine 1. Then, after S-VM1 performs configuration operations for the directly connected IO device 1, S-VM1 can directly access IO device 1.

[0272] Specifically, for example, S-VM1 can translate the GVA of the IO device 1 to be accessed into an IPA (for example, the IPA of IO device 1) through the first-stage page table created by S-VM1. The CPU uses the IPA to check the second-stage page table belonging to S-VM1 (generated by S305 in the above example 1). When the CPU determines that the IPA is mapped to an HPA in the second-stage page table, the CPU accesses the HPA to implement direct access of S-VM1 to IO device 1.

[0273] In this way, S-VM1 can access the HPA of the security device through GVA to achieve direct access to the security device without copying the access content between the non-secure area and the secure area.

[0274] In one possible implementation, whenever a confidential virtual machine is down (for example, the code executed by the CPU switches from a low privilege level (for example, S-EL0, S-EL1) to a high privilege level (for example, S-EL2)), for example, Figure 4d S401 in the example is a scenario where a confidential virtual machine is down. The S-visor may check the reason for the downfall of the confidential virtual machine and determine whether to forward the request of the confidential virtual machine to the service virtual machine or the N-visor based on the check result.

[0275] For example, S-VM1 makes an error in accessing IPA, causing S-VM1 to sink. IO devices and memory each have their own physical addresses (such as HPA). Usually, an IPA access error occurs because the HPA corresponding to the IPA does not exist in the second-stage page table belonging to S-VM1 in the S-visor (such as the second-stage page table configured by S305 in Example 3).

[0276] The S-visor can record the accessible device IPA range and the accessible memory IPA range of S-VM1. After S-VM1 falls, the S-visor can check whether the above-mentioned erroneous IPA belongs to the device IPA range or the memory IPA range.

[0277] If the erroneous IPA falls within the range of the above-mentioned device IPA, it indicates that the fall of S-VM1 is caused by S-VM1 accessing its directly connected security device. The S-visor may forward the erroneous request of S-VM1 (including the above-mentioned erroneous IPA) to the service virtual machine for processing.

[0278] If the erroneous IPA belongs to the above memory IPA range, it means that the sinking of S-VM1 is caused by S-VM1 accessing the memory. The S-visor can forward the erroneous request of S-VM1 (including the above erroneous IPA) to the N-visor for processing.

[0279] In the embodiment of the present application, the S-visor can correctly identify the processing object for the confidential virtual machine, whether to forward the request of the confidential virtual machine to the N-visor or the service virtual machine, thereby isolating the N-visor from the confidential virtual machine, isolating the confidential virtual machine from the service virtual machine, and ensuring the security of the confidential virtual machine.

[0280] The above embodiment describes how to pass through a security device (IO device 1 in the above example, and the same applies if there are multiple IO devices) to a confidential virtual machine. In other embodiments, when the S-visor receives a request from the S-VM (e.g., S-VM1) or N-visor to cancel the pass-through of a security device (e.g., IO device 1) to the S-VM1, similar to the principle of passing through a security device to a confidential virtual machine, the S-visor may, under the control of S-VM1 or N-visor, request the service virtual machine to cancel the pass-through of a security device that has been passed through to S-VM1. The service virtual machine may then notify the S-visor of the IPA of the security device to be passed through, and the S-visor may determine the MMIO mapped by the IPA by querying the second-stage page table obtained through S102. Then, the service virtual machine can restore the mapping of IPA and MMIO of the above-mentioned IO device 1 in the second-stage page table of the security device belonging to the S-VM1, add it to the second-stage page table belonging to the service virtual machine, and delete the mapping of IPA and MMIO of the above-mentioned IO device 1 in the second-stage page table of the security device belonging to the S-VM1. In this way, the S-visor can cancel the mapping of the physical address of IO device 1 to S-VM1 under the control of the service virtual machine, so that S-VM1 can no longer directly access IO device 1, and IO device 1 is restored to be directly managed and configured by the service virtual machine. The specific details will not be repeated here, and please refer to the introduction of Example 3.

[0281] In the above examples 1 to 4, the present application deploys the device management function in the service virtual machine (a confidential virtual machine), so that while avoiding the expansion of TCB (specifically S-visor), it can also manage the security devices in the secure world and directly configure the security devices of other confidential virtual machines through the service virtual machine.

[0282] The service virtual machine of the present application has the function of configuring the direct-pass device for the confidential virtual machine, avoiding the expansion of the TCB and ensuring a minimized TCB to reduce the attack plane. This allows the S-visor to focus only on security protection and decouple from specific virtualization functions as much as possible. In addition, the present application completely removes the virtualization overhead of the data plane through device direct-pass technology. Compared with PV I / O, the IO method that also requires the participation of the back-end driver is more efficient, and it also avoids the extra copies caused by shadow IO, which can greatly improve the IO performance of the confidential virtual machine.

[0283] Example 5, S-visor ensures secure communication between any two objects in N-visor, service virtual machine, and confidential virtual machine.

[0284] In the system of this application, please refer to Figure 3In the system of this application, the interaction between any two of the N-visor, the service virtual machine, and the confidential virtual machine needs to be implemented through the S-visor.

[0285] For example, a new service virtual machine is added to the system of the present application, and the service virtual machine can interact with the N-visor through the S-visor (for example, the process of registering the security device to the service virtual machine in Example 1). In addition, the service virtual machine can also interact with the confidential virtual machine through the S-visor (for example, the process of configuring a direct-pass security device for the confidential virtual machine in the service virtual machine in Example 3).

[0286] When any two objects in N-visor, service virtual machine, and confidential virtual machine interact through S-visor, the code executed by the CPU needs to switch from the code of one object to the code of another object, and when switching the codes of different objects, the CPU protects the CPU register scene by executing the S-visor code, where the S-visor code may include logic for performing scene protection of the CPU registers.

[0287] Figure 4e This is a schematic diagram of a scenario in which a service virtual machine interacts with a confidential virtual machine 1 through an S-visor. Figure 4e The system architecture can be referred to Figure 3 , I will not go into details here.

[0288] exist Figure 4e In the scenario, the S-visor receives request 1 from the service virtual machine at time t1, processes request 1, and sends request 2 related to request 1 (for example, forwarding the request to the service virtual machine) to the confidential virtual machine 1 at time t2. Then after time t1 and before time t2, that is, before the S-visor forwards request 2 to the confidential virtual machine 1, the S-visor can perform field protection on the CPU registers, such as clearing the value of the CPU registers to zero, or modifying the value of the CPU registers and other field protection schemes, to prevent the confidential virtual machine 1 from reading the data of the service virtual machine by reading the CPU registers after the code executed by the CPU switches from the code of the S-visor to the code of the confidential virtual machine 1, thereby ensuring secure communication between the confidential virtual machine 1 and the service virtual machine.

[0289] In addition, S-visor implements a solution to protect the CPU registers in the field (the principle can be found in Figure 4e), it can also be applied to the S-visor before forwarding the request of the confidential virtual machine to the service virtual machine. In this way, when the code executed by the CPU switches from the code of the confidential virtual machine to the service virtual machine, the S-visor can protect the code of the confidential virtual machine to prevent the service virtual machine from reading the security data of the confidential virtual machine and causing data leakage. In this way, the present application can use TCB (specifically S-visor) to check the relevant information of each sinking of the confidential virtual machine. Before forwarding the corresponding request outside the trusted domain, S-visor can hide the register scene of the confidential virtual machine to prevent information leakage. It can ensure that the state of the confidential virtual machine will not be leaked to the service virtual machine.

[0290] Similarly, the S-visor implements a solution to protect the CPU registers in context (the principle can be found in Figure 4e ), and can also be applied to the S-visor before forwarding the N-visor's request to the service virtual machine. In this way, when the code executed by the CPU switches from the N-visor's code to the service virtual machine, the S-visor can securely isolate the N-visor from the service virtual machine.

[0291] Similarly, the S-visor implements a solution to protect the CPU registers in context (the principle can be found in Figure 4e ), and can also be applied to the S-visor before forwarding the request of the service virtual machine to the N-visor. In this way, when the code executed by the CPU switches from the code of the service virtual machine to the N-visor, the S-visor can securely isolate the N-visor from the service virtual machine to ensure secure communication between the N-visor and the service virtual machine.

[0292] The S-visor's solution for protecting the CPU registers in the field is also applicable to the data interaction between the N-visor and the confidential virtual machine. The principle is the same as Figure 4e The example is similar to that of , so I will not repeat it here.

[0293] For example, the solution of S-visor performing on-site protection on CPU registers can be applied to any of the above examples 1 to 4, so that before S-visor forwards a request of an object A to another object B, S-visor can perform on-site protection on CPU registers to prevent data leakage caused by reading object A's data by object B, thereby ensuring secure communication between object A and object B. Wherein, object A is different from object B, and object A and object B can be any two objects in N-visor, service virtual machine, and S-visor.

[0294] This application uses the TCB (such as S-visor) in Secure World to ensure confidential interactions between confidential virtual machines and service virtual machines. For example, when the confidential virtual machine sinks to the S-visor, if the S-visor needs to be further forwarded to the service virtual machine for processing, in order to prevent the leakage of security data in the confidential virtual machine, the S-visor will hide the register scene of the confidential virtual machine. For another example, the modification of the second-stage page table of the confidential virtual machine by the service virtual machine needs to be checked by the S-visor before it can take effect. The confidential virtual machine can be isolated from the N-visor and the service virtual machine through the S-visor to prevent attacks from malicious hypervisors.

[0295] Example 6, secure memory configuration.

[0296] As mentioned above, TwinVisor's Split-CMA solution conflicts with the requirement for device direct pin memory. In addition, due to ARM hardware limitations, the secure world only supports up to 8 memory regions, so all confidential VMs can only use a total of 8 memory regions, which limits the secure memory accessible to confidential VMs on the host, further limiting the number of confidential VMs.

[0297] To this end, the present application provides a secure memory management method that can solve the above problems through a hybrid granularity secure memory management solution. The method can manage the secure memory of confidential virtual machines to solve the problem of the limited number of single-machine virtual machines caused by ARM hardware limitations.

[0298] Specifically, on the one hand, the present application adds a dedicated memory protection register (hereinafter referred to as per-cpu register, also referred to as first register) for each CPU to configure secure memory. Among them, the memory protection register can only be controlled by TCB (e.g. Figure 3 In one example, the present application can prevent the configuration of the secure memory from being tampered with by malicious attack code by limiting the privilege level of memory access. For example, the present application only allows code running in S-EL2 (e.g. Figure 3 S-visor in the CPU) to write per-cpu registers.

[0299] In addition, each CPU in the system of the present application may have its own per-cpu register, so that different memory address ranges can be configured for different CPUs through per-cpu registers, and mutually isolated secure memory configurations can be generated on different CPUs. In one example, Figure 3Different S-VMs can use different CPUs for memory access, and different CPUs can configure different secure memories through their per-cpu registers, thereby achieving memory isolation between S-VMs.

[0300] A CPU may have at least one set of per-cpu registers, wherein the set of per-cpu registers may be used to represent a memory address range, an organizational granularity (also called a protection mode), and at least one security attribute.

[0301] The CPU can have one or more sets of per-cpu registers. Each set of per-cpu registers can record a memory range. The memory range defined by multiple sets of per-cpu is the memory range accessible to the CPU.

[0302] For example, a group of per-cpu registers may include at least 4 per-cpu registers, and at least 4 per-cpu registers are used to record different attributes of a memory address range. The attributes are the base address of the memory area (referred to as Base attribute), the length of the memory area (referred to as Len attribute), the organizational granularity of the memory area (referred to as Mod attribute), and the security attribute of the memory area (referred to as Attr attribute).

[0303] Among them, the per-cpu register recording the Base attribute can record the base address of the memory address range;

[0304] The per-cpu register that records the Len attribute can record the length of the memory address range;

[0305] Then the per-cpu register that records the Base attribute and the per-cpu register that records the Len attribute can jointly record a memory address range.

[0306] The per-cpu register recording the Mod attribute can record whether the memory address range is a memory area (eg, Region) granularity or a memory page (eg, Page) granularity.

[0307] The memory address range of Region granularity may include one or more memory regions, and each memory region may include one or more memory pages.

[0308] The per-cpu register that records the Attr attribute can record the attributes of secure memory or normal memory.

[0309] In one scenario, when the Mod of a memory address range represented by a per-cpu register is Region granularity, and the value of the per-cpu register representing the record Attr attribute of the memory address range is Secure, then the memory address range represented by this group of per-cpu registers is secure memory.

[0310] In another scenario, when the Mod of a memory address range represented by a per-cpu register is Region granularity, and the value of the per-cpu register representing the record Attr attribute of the memory address range is normal, then the memory address range represented by this group of per-cpu registers is all non-secure memory (also called normal memory).

[0311] The memory address range of page granularity may include one or more memory pages.

[0312] When the Mod of a memory address range represented by a set of per-cpu registers is Page granularity, the number of per-cpu registers recording the Attr attribute can be multiple, and each per-cpu register here is used to record the security attributes (secure memory or normal memory) of different pages in the memory address range.

[0313] For example, in one scenario, when the Mod of a memory address range represented by a group of per-cpu registers is Page granularity, for example, the memory address range includes 3 memory pages, then the group of per-cpu registers may include 3 per-cpu registers for recording the Attr attribute, where the value of each per-cpu register for recording the Attr attribute may be Secure or normal, so that the 3 per-cpu registers for recording the Attr attribute are respectively used to record whether the security attribute of each of the above 3 memory pages is secure memory or normal memory.

[0314] In this way, when the protection mode of a memory address range represented by a set of per-cpu registers is Page granularity, each memory page in the memory address range can have its own security attribute (Attr attribute), and its security attribute can be Secure or normal, so that each memory page can be secure memory or non-secure memory. Then when the protection mode of a memory address range represented by a set of per-cpu registers is Page granularity, the memory address range represented by the set of per-cpu registers can include any number of memory pages belonging to secure memory, so that the number of secure memories accessed by confidential virtual machines is not limited to 8 from the hardware through the per-cpu registers of the CPU. In this way, the limitation problem that TZASC has a maximum of 8 Memory Regions can be solved at the hardware level, and the purpose of increasing the upper limit of the number of confidential virtual machines as much as possible under the premise of pinning memory is achieved.

[0315] In the embodiment of the present application, the memory area of ​​the secure memory is classified into Page granularity and Region granularity. The secure memory of Page granularity is a single memory page, which can minimize memory fragmentation and improve resource utilization. The secure memory of Region granularity can cover multiple memory pages. By accessing the secure memory of Region type, the confidential virtual machine can access multiple memory pages at one time, thereby reducing the access overhead of the secure memory.

[0316] In this way, the embodiment of the present application manages the secure memory accessed by the secure world through a hybrid method of Page granularity and Region granularity, thereby achieving a balance between memory access efficiency and memory utilization.

[0317] It should be understood that the above-mentioned embodiment uses the per-cpu register to record the attributes of a memory address range in the secure memory as an example. In other embodiments, the attributes of the ordinary memory can also be recorded using the per-cpu register. That is to say, the memory management method of the present application is not only applicable to the management of the secure memory accessed by the secure world, but also to the management of the ordinary memory of the ordinary world. The principles are similar and will not be repeated here.

[0318] On the other hand, in order to enable the confidential virtual machine of the secure world to access the memory address range of Page granularity, Figure 3The S-visor shown can configure the third-stage page table for memory devices (here, secure memory) under the control of the N-visor. The per-cpu register can point to the third-stage page table, which is a global page table for memory accessed by the CPU. It can distinguish CPUs, but does not distinguish between memory or secure devices accessible to the S-VM like the second-stage page table described above. In other words, different CPUs can have different third-stage page tables, and different S-VMs can not distinguish between the third-stage page tables when accessing memory devices, but directly use the third-stage page table used by the CPU of the S-VM to map the memory to achieve secure memory access.

[0319] Among them, the third-stage page table can be used to indicate the mapping relationship between the physical address of the secure memory (such as FPA) and whether the memory page belongs to the secure memory or the ordinary memory.

[0320] As described in Example 3 above, the S-visor can also configure the second-stage page table of the memory device belonging to S-VM1 under the control of the N-visor to limit the memory address range of the secure memory accessible to S-VM1, wherein the second-stage page table here can map the memory device accessible to S-VM1 ( Figure 4c The physical address range in the memory device shown in FIG. 1 . And the S-visor can, under the control of the N-visor, validate the second-stage page table to the register of the CPU (the specific method can refer to the prior art and is not limited here), so that the value of the register can point to the second-stage page table.

[0321] Similarly, when the granularity of a memory address range is Page granularity, the S-visor can also configure the third-stage page table for the memory address range under the control of the N-visor, and the S-visor applies the third-stage page table to the per-cpu register of the CPU so that the value of the per-cpu register of the CPU can point to the third-stage page table.

[0322] The configuration time of the third stage page table can be the system initialization stage or the S-VM memory access stage, which is not limited here.

[0323] The following takes the memory access stage of S-VM1 as an example to illustrate the configuration process of the page table in the third stage.

[0324] Combined with Figure 3 , Figure 4f For example, Figure 3 A diagram of the memory devices in .

[0325] like Figure 4fAs shown in (1), the memory device may include a secure memory and a normal memory, wherein the normal world can only access the normal memory, and the secure world can access the secure memory and the normal memory.

[0326] like Figure 4f As shown in (1), in the hybrid granularity memory management solution of the present application, the memory device is divided into nine memory areas, namely, memory area 41 to memory area 49.

[0327] The memory area 41 is a secure memory of Page granularity represented by a group of per-cpu registers, and the memory area 41 here is a memory page.

[0328] The memory area 42 is a section of ordinary memory with a Region granularity represented by a group of per-cpu registers, and the memory area 41 here includes multiple memory pages. The memory area 46 and the memory area 48 are similar, and will not be described here.

[0329] The memory area 43 is a secure memory of Region granularity represented by a set of per-cpu registers, and the memory area 43 here includes multiple memory pages. The memory area 45 and the memory area 49 are similar, and will not be described here.

[0330] The memory area 44 is a section of memory with Page granularity represented by a group of per-cpu registers. The memory area 44 here includes a memory page 12 belonging to the secure memory and a memory page 31 and a memory page 32 both belonging to the normal memory.

[0331] The memory area 47 is a section of memory with Page granularity represented by a group of per-cpu registers. The memory area 47 here includes three memory pages, all of which are secure memory, namely memory page 13, memory page 14, and memory page 15.

[0332] In some embodiments, reference may be made to Figure 3 , S-VM1 accesses secure memory page fault, causing N-visor to allocate a memory page through S-visor, such as allocating Figure 4f (1) shows that the memory page 31 of the normal memory is given to S-VM1. Then, N-visor controls S-visor to configure the memory page 31 as the secure memory belonging to S-VM1, for example, S-visor adds the mapping of the IPA to the FPA of the memory page 31 in the second-stage page table belonging to S-VM1, and makes the modified second-stage page table effective to the corresponding register of the CPU of S-VM1 (for details, please refer to the prior art, without limitation), so that the memory page 31 is added to the memory address range accessible to S-VM1.

[0333] In addition, under the control of the N-visor, the S-visor can also configure the third-stage page table for the memory page 31, so that the third-stage page table can include the FPA of the memory page 31 to the mapping indicating that the memory page 31 is secure memory, so that the memory page 31 is changed from ordinary memory to secure memory. Finally, the N-visor can control the S-visor to adjust the value of the corresponding group of per-cpu registers (a group of per-cpu registers including the address of the memory page 31 in the represented memory address range) of the above-mentioned CPU of the S-VM1, so that the information that the Attr attribute of the memory page 31 is Secure can take effect in the per-cpu register of the CPU.

[0334] When the S-visor configures the third-stage page table for the memory page 31, when the information of the memory page 31 is included in the third-stage page table of the CPU, the security attribute of the memory page 31 can be directly adjusted from ordinary memory to secure memory. When the information of the memory page 31 is not included in the third-stage page table of the CPU, a mapping relationship between the physical address of the memory page 31 and the memory page 31 as secure memory can be added in the third-stage page table.

[0335] In this way, the S-visor can translate the physical address (FPA) of the memory page into information about whether the memory page is secure memory through the third-stage page table, and write the register value pointing to the third-stage page table.

[0336] In this way, after the S-visor configures the third-stage page table for the memory page 31 allocated to S-VM1, and makes the third-stage page table effective in the per-cpu register of the CPU of S-VM1, then Figure 4f As shown in (2), the security attributes of memory page 31 are changed from Figure 4f (1) The normal memory shown is changed to Figure 4f (2) The secure memory shown.

[0337] Exemplarily, since each CPU has its own register for recording Base attributes (such as the above-mentioned register 1), different CPUs can have different secure memory views.

[0338] Exemplarily, before the CPU (also called virtual CPU, vcpu) of the confidential virtual machine runs, the S-visor may write the information of the second-stage page table indicating the memory address range stored in the data structure of the vcpu and the information indicating the third-stage page table into the corresponding registers of the vcpu to correctly complete the mapping of the memory accessible to the confidential virtual machine.

[0339] In this way, when the S-visor configures secure memory for S-VM1, when the secure memory allocated to S-VM1 by N-visor is at Page granularity, the S-visor adds the mapping of the physical address of the allocated memory page to the second-stage page table belonging to S-VM1, so that S-VM1 can access the allocated memory page; and the S-visor can adjust the security attribute of the memory page from ordinary memory to secure memory by modifying the third-stage page table, and adjust the value of the per-cpu register related to the memory page. In this way, the present application can flexibly manage discrete secure memory through the third-stage page table, and the page-granular memory configuration can also improve memory utilization.

[0340] Similarly, when the memory area allocated to S-VM1 by N-visor through S-visor is of Region granularity, but only one memory page is mapped to S-VM1 at a time, S-visor can add a mapping of the physical address of the memory page in the second-stage page table belonging to S-VM1; and S-visor can adjust the security attribute of the per-cpu register related to the memory page (wherein the memory range represented by the per-cpu register includes the memory page), so that the security attribute of the memory range represented by the per-cpu register is adjusted from ordinary memory to secure memory, and the organizational granularity of the memory range represented by the per-cpu register is Region granularity. Among them, since the organizational granularity of the allocated memory area is Region granularity, S-visor does not need to adjust the third-stage page table, but only needs to modify the second-stage page table belonging to the S-VM1, and make the modified second-stage page table effective to the corresponding register of the S-VM1, and modify the security attribute of the corresponding per-cpu register.

[0341] When the memory area allocated by N-visor to S-VM1 through S-visor is of Region granularity, other processes are similar to the principle of allocating memory areas of Page granularity, which will not be repeated here.

[0342] Similarly, S-visor can also be controlled by N-visor. Figure 3 The memory area in the memory device shown is changed from secure memory to normal memory. The specific principle is similar to the above-mentioned process of changing the memory area 31 from normal memory to secure memory, which will not be repeated here.

[0343] In this way, when the memory allocated to the S-VM is at the Page granularity, the S-visor can, under the control of the N-visor, add the mapping of the physical address of the corresponding memory page in the second-stage page table belonging to the S-VM, and change the security attribute of the memory page through the third-stage page table, so that the memory page can be switched between secure memory and ordinary memory, and the modified third-stage page table is effective in the per-cpu register of the corresponding CPU. When the memory allocated to the S-VM is at the Region granularity, the S-visor can, under the control of the N-visor, add the mapping of the physical address of the corresponding memory page in the second-stage page table belonging to the S-VM, and make the memory page effective as secure memory in the corresponding per-cpu register.

[0344] Example 7, secure memory access.

[0345] In scenario 1, the memory accessed is at the Page granularity.

[0346] For reference Figure 3 and Figure 4f (2) When a confidential virtual machine (e.g., S-VM1) accesses memory, S-VM1 can translate the GVA of the memory to be accessed into IPA through the first-stage page table. The CPU uses the IPA to check the second-stage page table of the memory device belonging to S-VM1. The CPU can determine the memory physical address mapped by the IPA in the second-stage page table, which means that the memory accessed by S-VM1 is within the authorized memory address range. Then, the CPU reads the per-cpu register representing the memory address range (wherein the above-mentioned mapped memory physical address belongs to the memory address range). For example, the organizational granularity recorded by the per-cpu register used to record the organizational granularity in the group of per-cpu registers is Page, which means that the memory currently accessed is Page granularity. Then, the CPU can query the third-stage page table to query the information mapped by the above-mentioned memory physical address (here, the physical address of a memory page) after the second-stage page table translation, which is whether the memory page is secure memory or ordinary memory. Finally, the CPU uses the mapping information and the above memory physical address to determine memory access or error reporting to improve security. For specific details, please refer to the relevant technology, and no details are limited here.

[0347] In the above scenario 1, when the object accessing the memory is a normal virtual machine, if the CPU determines that the memory page corresponding to the above memory physical address currently accessed is secure memory by querying the third-stage page table, the CPU can reject the memory access request of the normal virtual machine.

[0348] In scenario 2, the memory accessed is at the Region granularity.

[0349] For reference Figure 3and Figure 4f (2) When a confidential virtual machine (such as S-VM1) accesses memory, S-VM1 can translate the GVA of the memory to be accessed into IPA through the first-stage page table. The CPU uses the IPA to check the second-stage page table of the memory device belonging to S-VM1. The CPU can determine the memory physical address (such as HPA) mapped by the IPA in the second-stage page table, which means that the memory accessed by S-VM1 is within the authorized memory address range. Then, the CPU reads the per-cpu register representing the memory address range (wherein the above-mentioned mapped memory physical address belongs to the memory address range). For example, the organizational granularity recorded by the per-cpu register used to record the organizational granularity in the group of per-cpu registers is Region, which means that the memory currently accessed is Region granularity, and the CPU does not need to query the third-stage page table.

[0350] Among them, when the CPU reads the per-cpu register representing the memory address range (wherein the above-mentioned mapped memory physical address belongs to the memory address range), if the value of the per-cpu register used to record the security attribute is read as Secure, the CPU can access the memory according to the HPA; if the value of the per-cpu register used to record the security attribute is read as normal, the CPU can access the memory according to the HPA, or report an error. No detailed restrictions are made here.

[0351] In the above scenario 2, when the object accessing the memory is a normal virtual machine, the CPU can read the per-cpu register representing the memory address range (wherein the above-mapped memory physical address belongs to the memory address range) after querying the second-stage page table. If the value of the per-cpu register used to record the security attribute is read as Secure, the CPU can reject the memory access request of the normal virtual machine. If the value of the per-cpu register used to record the security attribute is read as normal, the CPU can access the memory physical address of the memory device after translation by the second-stage page table.

[0352] In a possible implementation, the operating system can dynamically fill in the value of the per-cpu register according to current needs. For example, if the current Normal World memory resources are tight, that is, the normal memory is less or insufficient, then the CPU can map some idle memory pages belonging to the secure memory to the normal memory in the third-stage page table, so that these memory pages are changed from secure memory to normal memory, and the free secure memory pages are released for use by the N-visor to alleviate the tight memory resources in the normal world.

[0353] For another example, if the current Normal Memory is not tight and the OS wants to improve the memory access efficiency, the system of the present application can merge some continuous memory pages into the same Region.

[0354] This application improves the CPU hardware so that it supports multiple per-cpu registers to store metadata of secure memory, thus solving the Memory Region limitation problem caused by ARM TZASC from the root.

[0355] This application also proposes a management and access solution for secure memory with hybrid granularity. For Page granularity, a third-stage page table will be added based on the virtualized second-stage page table. The CPU can improve memory utilization by traversing the third-stage page table to find out whether the final physical address is secure memory or ordinary memory. For Region granularity, a register describes a memory area. After traversing the second-stage page table, the CPU can directly check the per-cpu register to determine the security attributes of the memory area. If the security attributes are within the access rights of the object accessing the memory, memory can be accessed directly to improve memory range efficiency without reading the third-stage page table. Hybrid-granular secure memory protection technology can maximize the number of single-machine confidential virtual machines under the premise of pinning memory, and achieve a reasonable trade-off between memory access overhead and resource utilization through the configuration of Page Based and Region Based granularities.

[0356] The CPU of the present application supports per-cpu registers in hardware, and can read the third-stage page table for memory access when the memory accessed is at Page granularity, wherein the S-visor implements the configuration, storage and use of the third-stage page table in software.

[0357] As described above, there are two main solutions for confidential virtualization of the ARM platform, one is to use TwinVisor, and the other is to use hardware-extended ARM CCA. The above embodiment takes TwinVisor as an example to introduce the method and system of the present application. In other embodiments, the method can also be applied to ARM CCA.

[0358] Specifically, ARM CCA introduces Realm World in addition to Normal World and Secure World. The RMM in Realm World is similar to N-visor in TwinVisor. Therefore, similar to TwinVisor, ARM CCA's system software architecture also has two Hypervisors.

[0359] Each of the above embodiments proposed in this application can be applied to the ARMCCA platform.

[0360] For example, the software and hardware combination solution of the present application can provide a feasible implementation solution for device pass-through of the ARM CCA platform. ARM CCA can add per-cpu registers to support a mixed-granularity secure memory protection mechanism. And the present application can run Tri-visor in the realm world of ARM CCA to support the management of secure devices and pass-through configuration. And, the present application can also hide the register scene in RMM to ensure the secure communication between the confidential virtual machine and the Tri-visor. For example, RMM can check the sinking of the confidential virtual machine and forward the Tri-Visor to the Hypervisor (such as N-visor) in NormalWorld after hiding the virtual machine register scene.

[0361] The method of the present application is applied to ARM CCA, and Tri-Visor can be run in a virtual machine, reusing the mature device management capabilities already in Linux, streamlining RMM, and reducing the attack surface. It can also provide mixed-granularity secure memory protection, combining memory access efficiency with management granularity. It can also correctly identify the processing object of the virtual machine sinking, ensuring data security in confidential virtual machines. The design of per-cpu registers allows for more flexible and precise restrictions on secure memory access rights on different CPUs.

[0362] Furthermore, the idea of ​​using a specific virtual machine as a server to provide services can be applied to more scenarios. For example, the Intel VMFUNC instruction can switch the EPT page table of the virtual machine. By changing the EPT, the memory view obtained by the virtual machine can be changed, so as to quickly switch between virtual machines. Many services in the microkernel operating system (such as network services, file system services, etc.) are implemented as dedicated user-mode threads. However, when requesting ipc services, the client thread needs to sink to the kernel first, and the context switch to the server thread is completed. Frequent sinks and context switches may bring high performance overhead. If the relevant services can be placed in the virtual machine, quickly switching to the server through the vmfunc instruction may bring a certain degree of performance improvement.

[0363] At the same time, the method of adding per-cpu registers to the CPU can also be applied to other scenarios. For example, in confidential virtualization, the hypervisor cannot access the encrypted memory of the confidential virtual machine. If a series of registers can be added to point to the memory address that the guest wants to share with the hypervisor, the performance overhead caused by the extra copy inside the guest can be avoided.

[0364] The following describes a device provided by an embodiment of the present application. Figure 5 As shown:

[0365] Figure 5 A schematic diagram of the structure of a resource control device provided in an embodiment of the present application. Figure 5 As shown, the device 500 may include: a processor 501 , a transceiver 505 , and optionally a memory 502 .

[0366] The transceiver 505 may be referred to as a transceiver unit, a transceiver, or a transceiver circuit, etc., and may be used to implement a transceiver function. The transceiver 505 may include a receiver and a transmitter, wherein the receiver may be referred to as a receiver or a receiving circuit, etc., and may be used to implement a receiving function; and the transmitter may be referred to as a transmitter or a transmitting circuit, etc., and may be used to implement a transmitting function.

[0367] The memory 502 may store a computer program or software code or instruction 504, which may also be referred to as firmware. The processor 501 may implement the resource control method provided in each embodiment of the present application by running the computer program or software code or instruction 503 therein, or by calling the computer program or software code or instruction 504 stored in the memory 502. The processor 501 may be a central processing unit (CPU), and the memory 502 may be, for example, a read-only memory (ROM) or a random access memory (RAM).

[0368] The processor 501 and transceiver 505 described in the present application can be implemented on an integrated circuit (IC), an analog IC, a radio frequency integrated circuit RFIC, a mixed signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc.

[0369] The above-mentioned device 500 may further include an antenna 506. The modules included in the device 500 are only for illustration and are not limited in this application.

[0370] For example, the structure of the resource control device may not be affected by Figure 5 The resource control device may be an independent device or may be part of a larger device. For example, the resource control device may be implemented in the form of:

[0371] (1) An independent integrated circuit IC, or chip, or chip system or subsystem; (2) A collection of one or more ICs, optionally, the IC collection may also include a storage component for storing data and instructions; (3) A module that can be embedded in other devices; (4) In-vehicle equipment, etc.; (5) Others, etc.

[0372] For the case where the resource control device is implemented in the form of a chip or a chip system, see Figure 6 Schematic diagram of the chip structure shown. Figure 6 The chip shown includes a processor 601 and an interface 602 . The number of the processor 601 may be one or more, and the number of the interface 602 may be multiple. Optionally, the chip or chip system may include a memory 603 .

[0373] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0374] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes at least one code segment, and the at least one code segment can be executed by a computer to control the computer to implement the above method embodiment.

[0375] Based on the same technical concept, the embodiment of the present application also provides a computer program, which, when executed, is used to implement the above method embodiment.

[0376] The program may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor.

[0377] Based on the same technical concept, the embodiment of the present application further provides a chip, including a processor. The processor can implement the above method embodiment.

[0378] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disk (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0379] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0380] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A resource control method, characterized in that: Applied to a resource control device, the resource control device includes a device management module and a communication module, wherein both the device management module and the communication module run in a trusted execution environment TEE, and the privilege level of the communication module is higher than the privilege level of the device management module, the method includes: The device management module sends first information to the communication module, wherein the first information includes identification information indicating the first IO device; The communication module acquires a first physical address range of the first IO device based on the first information; The communication module associates the first physical address range with a first secure virtual machine, wherein the first secure virtual machine runs in the TEE and a privilege level of the first secure virtual machine is lower than a privilege level of the communication module.

2. The method according to claim 1, characterized in that The device management module runs in the second secure virtual machine.

3. The method according to claim 1 or 2, characterized in that: The resource control device further includes a processing module, and after the communication module associates the first physical address range with the first secure virtual machine, the method further includes: The processing module queries the first physical address range associated with the first secure virtual machine according to the first access address, and determines a second access address mapped to the first access address; The first access address is a resource access address in an instruction currently executed by the first secure virtual machine, and the second access address is a physical address in the first physical address range; The processing module accesses the target resource in the first IO device according to the second access address.

4. The method according to any one of claims 1 to 3, characterized in that The device management module includes identification information of at least one second IO device, and the at least one second IO device includes the first IO device.

5. The method according to claim 4, characterized in that Before the device management module sends the first information to the communication module, the method further includes: The communication module associates a second physical address range of the at least one second IO device with the device management module.

6. The method according to claim 5, characterized in that The communication module acquires a first physical address range of the first IO device based on the first information, including: The communication module obtains a first physical address range of the first IO device within the second physical address range associated with the device management module based on the first information, wherein the second physical address range includes the first physical address range.

7. The method according to any one of claims 2 to 6, characterized in that After the communication module acquires the first physical address range of the first IO device based on the first information, the method further includes: The communication module cancels the association between the first physical address range and the second secure virtual machine.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: The device management module sends a first request to the communication module; The communication module hides the value of the register of the resource control device before issuing the second request; The second request is a request after the communication module processes the first request.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: The communication module receives a third request from the first secure virtual machine; Before the communication module issues the fourth request, the communication module hides the value of the register of the resource control device; The fourth request is a request after the communication module processes the third request.

10. The method according to any one of claims 1 to 9, characterized in that After the communication module associates the first physical address range with the first secure virtual machine, the method further includes: The communication module determines, according to the third access address, that the first physical address range associated with the first secure virtual machine does not include a physical address mapped to the third access address within the first physical address range; The third access address is a resource access address in a first access request sent by the first secure virtual machine to the communication module; The communication module refuses to forward the first access request.

11. The method according to any one of claims 3 to 10, characterized in that The number of the processing modules is at least one, and the processing module includes at least one group of first registers, each group of the first registers is used to record a third physical address range of a memory resource, the organizational granularity of the memory resource, and at least one security attribute of the memory resource.

12. The method according to claim 11, characterized in that The organizational granularity is a region granularity or a page granularity, wherein the memory resources of the region granularity include at least one memory region, and the memory resources of the page granularity include at least one memory page.

13. The method according to claim 12, characterized in that The security attribute is a first attribute representing secure memory or a second attribute representing ordinary memory, wherein the memory resource with the security attribute being the first attribute is only accessed in the TEE environment, and the security attributes of each memory page in the page-granularity memory resources are independent of each other, and the security attributes of each memory area in the area-granularity memory resources are the same.

14. The method according to any one of claims 11 to 13, characterized in that The method further comprises: The processing module configures the value of the at least one group of first registers according to the first instruction; The first instruction is the instruction currently executed by the communication module.

15. The method according to any one of claims 11 to 14, characterized in that The method further comprises: The communication module associates a first physical address of a first memory page with information indicating that the security attribute of the memory page is the first attribute; The first memory page is a memory page associated with the first physical address and the first secure virtual machine; The processing module modifies the value of the second register in the first register to a value indicating the first attribute according to the second instruction; The second register is a first register in the group of first registers for recording the security attribute of the first memory page; The second instruction is the instruction currently executed by the communication module.

16. The method according to any one of claims 11 to 15, characterized in that The method further comprises: The communication module associates the second physical address of the second memory page with the first secure virtual machine; The processing module modifies the value of the third register in the first register to a value indicating the first attribute according to the third instruction; The third register is a first register in the group of first registers for recording the security attribute of the first memory area, wherein the first memory area includes the second memory page; The third instruction is the instruction currently executed by the communication module.

17. A resource control device, characterized in that: The resource control device comprises: a device management module and a communication module, wherein both the device management module and the communication module run in a trusted execution environment TEE, and the privilege level of the communication module is higher than the privilege level of the device management module; The device management module is used to send first information to the communication module, wherein the first information includes identification information indicating the first IO device; The communication module is configured to obtain a first physical address range of the first IO device based on the first information; The communication module is used to associate the first physical address range with a first secure virtual machine, wherein the first secure virtual machine runs in the TEE and a privilege level of the first secure virtual machine is lower than a privilege level of the communication module.

18. The device according to claim 17, characterized in that The device management module runs in the second secure virtual machine.

19. The device according to claim 17 or 18, characterized in that The resource control device also includes a processing module; The processing module is used to query the first physical address range associated with the first secure virtual machine according to the first access address, and determine the second access address mapped to the first access address; The first access address is a resource access address in an instruction currently executed by the first secure virtual machine, and the second access address is a physical address in the first physical address range; The processing module is further configured to access a target resource in the first IO device according to the second access address.

20. The device according to any one of claims 17 to 19, characterized in that The device management module includes identification information of at least one second IO device, and the at least one second IO device includes the first IO device.

21. The device according to claim 20, characterized in that The communication module is further used to associate the second physical address range of the at least one second IO device with the device management module.

22. The device according to claim 21, characterized in that The communication module is specifically configured to obtain, based on the first information, a first physical address range of the first IO device within the second physical address range associated with the device management module, wherein the second physical address range includes the first physical address range.

23. The device according to any one of claims 18 to 22, characterized in that The communication module is further used to cancel the association between the first physical address range and the second secure virtual machine.

24. The device according to any one of claims 17 to 23, characterized in that The device management module is further configured to send the first request to the communication module; The communication module is further used to hide the value of the register of the resource control device before issuing the second request; The second request is a request after the communication module processes the first request.

25. The device according to any one of claims 17 to 24, characterized in that The communication module is further configured to receive a third request from the first secure virtual machine; The communication module is further configured to hide the value of the register of the resource control device before issuing the fourth request; The fourth request is a request after the communication module processes the third request.

26. The device according to any one of claims 17 to 25, characterized in that The communication module is further configured to determine, according to the third access address, by querying the first physical address range associated with the first secure virtual machine, that the first physical address range does not include a physical address mapped to the third access address; The third access address is a resource access address in a first access request sent by the first secure virtual machine to the communication module; The communication module is further configured to refuse to forward the first access request.

27. The device according to any one of claims 19 to 26, characterized in that The number of the processing modules is at least one, and the processing module includes at least one group of first registers, each group of the first registers is used to record a third physical address range of a memory resource, the organizational granularity of the memory resource, and at least one security attribute of the memory resource.

28. The device according to claim 27, characterized in that The organizational granularity is a region granularity or a page granularity, wherein the memory resources of the region granularity include at least one memory region, and the memory resources of the page granularity include at least one memory page.

29. The device according to claim 28, characterized in that The security attribute is a first attribute representing secure memory or a second attribute representing ordinary memory, wherein the memory resource with the security attribute being the first attribute is only accessed in the TEE environment, and the security attributes of each memory page in the page-granularity memory resources are independent of each other, and the security attributes of each memory area in the area-granularity memory resources are the same.

30. The device according to any one of claims 27 to 29, characterized in that The processing module is further configured to configure the value of the at least one group of first registers according to the first instruction; The first instruction is the instruction currently executed by the communication module.

31. The device according to any one of claims 27 to 30, characterized in that The communication module is further used to associate the first physical address of the first memory page with information indicating that the security attribute of the memory page is the first attribute; The first memory page is a memory page associated with the first physical address and the first secure virtual machine; The processing module is further configured to modify the value of the second register in the first register to a value indicating the first attribute according to the second instruction; The second register is a first register in the group of first registers for recording the security attribute of the first memory page; The second instruction is the instruction currently executed by the communication module.

32. The device according to any one of claims 27 to 31, characterized in that The communication module is further used to associate a second physical address of a second memory page with the first secure virtual machine; The processing module is further configured to modify the value of the third register in the first register to a value indicating the first attribute according to a third instruction; The third register is a first register in the group of first registers for recording the security attribute of the first memory area, wherein the first memory area includes the second memory page; The third instruction is the instruction currently executed by the communication module.

33. A computer-readable storage medium, characterized in that: The method comprises a computer program, which, when running on a computer or a processor, enables the computer or the processor to execute the method according to any one of claims 1 to 16.

34. A resource control device, characterized in that: It includes one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the processor is used to execute the method as claimed in any one of claims 1 to 16.

35. A computer program product, characterized in that The computer program product comprises a software program, and when the software program is executed by a computer or a processor, the steps of the method according to any one of claims 1 to 16 are performed.

Images