Apparatus for providing amd sev-based trusted execution environment and trusted execution system

Abstract

Embodiments of the present specification provide an AMD SEV-based trusted execution environment architecture and a trusted execution system. The trusted execution environment architecture comprises: an instruction emulator, which hooks a non-legal opcode trap handler in the kernels of both an Enclave virtual machine and an application virtual machine to emulate execution of SGX instructions; an Enclave manager, which is deployed in the Enclave virtual machine and is configured to create and manage Enclave applications; a memory manager, which manages internal memory of the Enclave virtual machine to allow an Enclave application to access internal memory of the Enclave virtual machine and external memory of the Enclave virtual machine while prohibiting external components of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and a cross-virtual machine communicator, which enables cross-virtual machine communication between the Enclave virtual machine and the application virtual machine.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification generally relate to the field of trusted execution, and more particularly, to an AMD SEV based trusted execution environment architecture and trusted execution system. BACKGROUND

[0002] In recent years, the use of Trusted Execution Environments (TEE) such as Intel SGX (Software Guard Execution) and AMD SEV (Secure Encrypted Virtualization) has grown rapidly. TEEs can protect the confidentiality and integrity of program code and data from attacks by malicious system software, making it very valuable to deploy TEEs in cloud systems where the computing platform is not fully trusted by the customer.

[0003] SGX, as a mainstream TEE platform, has become a de facto standard for building TEE based applications. However, the Instruction Set Architecture (ISA) extensions of SGX explicitly separate software applications into trusted and untrusted software applications, such that trusted software applications are isolated within a protected Enclave region that is only accessible by code executing in Enclave mode. Similarly, developers need to refactor existing applications or build new applications according to the SGX software specification. And the applications are compiled using the SGX SDK. As a result, applications developed for SGX can only run on SGX processors, leading to a vendor lock-in situation. SUMMARY

[0004] In view of the above, embodiments of the present specification provide an AMD SEV based trusted execution environment architecture and trusted execution system. With the trusted execution environment architecture, TEE applications can be decoupled from underlying TEE hardware, such that the trusted execution environment architecture can provide binary code compatibility for partitioned SGX Enclave software, and can directly execute binary code of SGX Enclave software on AMD SEV.

[0005] According to an aspect of the embodiments of the present specification, there is provided an AMD SEV-based trusted execution environment architecture, comprising: an instruction emulator, hooking an illegal opcode trap handler in the kernel of both an Enclave virtual machine and an application virtual machine to emulate execution of SGX instructions; an Enclave manager, deployed in the Enclave virtual machine, for creating and managing Enclave applications; a memory manager, managing internal memory of the Enclave virtual machine to allow the Enclave applications to access internal memory of the Enclave virtual machine and external memory of the Enclave virtual machine, while prohibiting external components of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and a cross-virtual machine communicator, enabling cross-virtual machine communication between the Enclave virtual machine and the application virtual machine.

[0006] Optionally, in one example of the above aspect, the trusted execution environment architecture can further comprise: a remote authenticator, deployed in the underlying AMD SEV hardware structure and the kernel of the Enclave virtual machine, for remote authentication of Enclave applications based on an AMD SEV-based remote authentication framework.

[0007] Optionally, in one example of the above aspect, the instruction emulator is configured to emulate execution of all ENCLS instructions on the boundary between the Enclave virtual machine and the application virtual machine; emulate execution of ENCLU instructions other than EENTER, EEXIT and ERESUME instructions inside the Enclave virtual machine according to hardware specifications; and emulate execution of the EENTER, EEXIT and ERESUME instructions across virtual machines.

[0008] Optionally, in one example of the above aspect, the instruction emulator is configured to emulate execution of all ENCLS instructions in the following manner: package parameters of the ENCLS instructions and necessary data into a request data packet and send to the Enclave virtual machine for execution; and return execution results as a response to the application virtual machine after the Enclave virtual machine completes emulated instruction execution.

[0009] Optionally, in one example of the above aspect, when the ENCLS instruction is not an EWB instruction, the execution results contain an error code and register values that need to be updated, and when the ENCLS instruction is an EWB instruction, the execution results contain encrypted Enclave pages and their metadata.

[0010] Optionally, in one example of the above aspect, when the ENCLS instruction is an EWB instruction and the EWB instruction execution fails, the response does not contain a page payload and metadata.

[0011] Optionally, in one example of the above aspect, the instruction emulator is configured to emulate the EENTER instruction, the EEXIT instruction and the ERESUME instruction in the following manner: intercepting an EENTER instruction via a first illegal opcode trap handler at a kernel of the application virtual machine, preparing corresponding EENTER instruction parameters, generating an EENTER request and sending the EENTER request to an EENTER handler at the Enclave virtual machine, the EENTER request including the EENTER instruction parameters, and suspending an execution thread of an application at the application virtual machine to wait for a corresponding EEXIT; in response to receiving the EENTER request, obtaining a corresponding Enclave thread for processing via an Enclave hypervisor at the Enclave virtual machine; when an EEXIT instruction is executed at the Enclave virtual machine, intercepting the EEXIT instruction via a hook at a second illegal opcode trap handler at a kernel of the Enclave virtual machine, generating an EEXIT request, and sending the EEXIT request to the first illegal opcode trap handler at the application virtual machine, in response to receiving the EEXIT request, resuming execution of the execution thread of the application via an EEXIT handler at the application virtual machine.

[0012] Optionally, in one example of the above aspect, the instruction emulator is further configured to return a processing result of the EENTER request as an EENTER response to the first illegal opcode trap handler at the application virtual machine via the EENTER handler

[0013] Optionally, in one example of the above aspect, the instruction emulator is further configured to terminate execution of the Enclave thread in response to the second illegal opcode trap handler intercepting the EEXIT instruction.

[0014] Optionally, in one example of the above aspect, the instruction emulator is configured to trigger the asynchronous Enclave exit via the error in the following manner: checking, via a hook at a second illegal opcode trap handler of a kernel of the Enclave virtual machine, whether the error is triggered by an Enclave thread; in response to the error being triggered by the Enclave thread, invoking an asynchronous Enclave exit handler via the second illegal opcode trap handler, and putting the Enclave thread into a sleep mode; generating, via the asynchronous Enclave exit handler, a synthetic register state, and sending the synthetic register state to a first illegal opcode trap handler at the application virtual machine; waking up, via the first illegal opcode trap handler, an execution thread of an application at the application virtual machine to execute error handling using the synthetic register state by invoking a corresponding error handler within an illegal opcode trap context of a previous EENTER instruction; in response to completing the error handling, re-executing, via the first illegal opcode trap handler, a re-execution handler, and the second illegal opcode trap handler, the Enclave thread at the Enclave virtual machine using an execution thread pointed to by an asynchronous exit pointer to re-execute the process.

[0015] Optionally, in one example of the above aspect, the Enclave manager is configured to: create an Enclave context dedicated for Enclave execution, the Enclave context being separate and isolated from a manager context of the Enclave manager; configure a memory synchronization thread and a dispatch thread, and put the memory synchronization thread and the dispatch thread into the Enclave context by trapping a kernel code of the Enclave virtual machine to perform an action; register itself as an idle Enclave manager to a kernel to wait for an EENTER request for creating an Enclave thread; in response to receiving the EENTER request, create the Enclave thread for executing Enclave code; and switch a context of the Enclave thread to the Enclave context after the Enclave thread is properly set up and detached from the manager thread.

[0016] Optionally, in one example of the above aspect, a kernel interface of the Enclave virtual machine is set such that the Enclave thread is not allowed to use any system call instruction or software interrupt, and the Enclave context is not mapped to any system call instruction.

[0017] Optionally, in one example of the above aspect, the memory manager includes a first memory manager deployed in a kernel of the Enclave virtual machine and a second memory manager deployed in a kernel of the application virtual machine. The memory manager is configured to manage virtual address mapping and access permissions for each EPC page using a software Enclave Page Cache (EPC) mapping table, each entry of the software EPC mapping table storing virtual address mapping and access permissions for each EPC page; enable Enclave code running in the Enclave virtual machine to access memory in the application virtual machine using a fetch and map mechanism hooked with a page fault handler of the Enclave virtual machine; and maintain page synchronization across the Enclave virtual machine and the application virtual machine using a no-switch synchronization mechanism.

[0018] Optionally, in one example of the above aspect, the memory manager is configured to prohibit performing fetch and map operations on Enclave pages whose virtual addresses fall within an Enclave linear address range.

[0019] Optionally, in one example of the above aspect, the address space of the trusted execution environment architecture includes an EPC address space, a virtual address space, an Enclave physical address space, and a manager address space. The manager address space is managed by a kernel of the Enclave virtual machine, the virtual address space is shared by an application in the application virtual machine and an Enclave thread in the Enclave virtual machine, the EPC address is used as a physical address in the application virtual machine and as an index to the EPCM in the Enclave virtual machine, the Enclave physical address is a backup of Enclave memory mapped to a virtual address in the Enclave virtual machine and is used to perform addressing.

[0020] Optionally, in one example of the above aspect, the cross-VM communicator includes a first cross-VM communicator deployed in a kernel of the Enclave VM, a second cross-VM communicator deployed in a kernel of the application VM, and a hub deployed in a hypervisor. The cross-VM communicator is configured to perform cross-VM communication between the Enclave VM and the application VM using a cross-VM communication protocol as follows: preparing a data packet to be transmitted in a shared memory via a sender thread in a first VM, and signaling a CPUID handler in the hub using a CPUID instruction whose argument indicates a virtual machine ID of a target second VM; picking up the data packet in the shared memory via the CPUID handler, and asking a send worker thread in the hub to notify the target second VM via an IRQ handler; notifying the hypervisor using the CPUID instruction via the IRQ handler, and fetching the data packet back to a shared memory of the target second VM; sending the data packet to a dispatcher of the target second VM through a dispatch queue of the target second VM; and decrypting, stitching, and sending the stitched data table to a destination handler at the target second VM via the dispatcher.

[0021] Optionally, in one example of the above aspect, the remote authenticator uses a public key provided by a trusted execution environment architecture provider and a root key stored in a kernel of the Enclave VM to emulate a key fetching instruction of SGX to fetch a verification key to implement remote authentication.

[0022] According to another embodiment of the present specification, a trusted execution system is provided, comprising: a trusted execution environment architecture as described above; an Enclave VM; an application VM; a hypervisor; and an underlying AMD SEV hardware structure.

[0023] According to another embodiment of the present specification, there is provided an AMD SEV based trusted execution environment architecture, comprising: at least one processor, a memory coupled with the at least one processor, and a computer program stored in the memory, the at least one processor executing the computer program to implement: hooking an illegal opcode trap handler in the kernel of both an Enclave virtual machine and an application virtual machine to emulate execution of SGX instructions; creating and managing an Enclave application; managing internal memory of the Enclave virtual machine to allow the Enclave application to access internal memory of the Enclave virtual machine and external memory of the Enclave virtual machine while prohibiting external components of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and implementing cross- virtual machine communication between the Enclave virtual machine and the application virtual machine.

[0024] According to another embodiment of the present specification, there is provided a computer readable storage medium storing executable instructions which, when executed, cause a processor to perform: hooking an illegal opcode trap handler in the kernel of both an Enclave virtual machine and an application virtual machine to emulate execution of SGX instructions; creating and managing an Enclave application; managing internal memory of the Enclave virtual machine to allow the Enclave application to access internal memory of the Enclave virtual machine and external memory of the Enclave virtual machine while prohibiting external components of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and implementing cross- virtual machine communication between the Enclave virtual machine and the application virtual machine.

[0025] According to another embodiment of the present specification, there is provided a computer program product comprising a computer program which, when executed by a processor, implements: hooking an illegal opcode trap handler in the kernel of both an Enclave virtual machine and an application virtual machine to emulate execution of SGX instructions; creating and managing an Enclave application; managing internal memory of the Enclave virtual machine to allow the Enclave application to access internal memory of the Enclave virtual machine and external memory of the Enclave virtual machine while prohibiting external components of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and implementing cross- virtual machine communication between the Enclave virtual machine and the application virtual machine. BRIEF DESCRIPTION OF DRAWINGS

[0026] A further understanding of the nature and advantages of the contents of this specification can be realized by reference to the following drawings. In the drawings, like components or features can have the same reference label.

[0027] Figure 1 An example diagram of an AMD SEV-based trusted execution system is shown in accordance with an embodiment of the present specification.

[0028] Figure 2 SGX instructions that can be supported by a trusted execution environment architecture are shown in accordance with an embodiment of the present specification.

[0029] Figure 3 An example flow diagram of an ENCLS instruction emulation process is shown in accordance with an embodiment of the present specification.

[0030] Figure 4 An example flow diagram of a cross-VM emulation of EENTER, EEXIT, and ERESUMW instructions is shown in accordance with an embodiment of the present specification.

[0031] Figure 5 An example flow diagram of an asynchronous enclave exit triggering process is shown in accordance with an embodiment of the present specification.

[0032] Figure 6 An example flow diagram of an enclave application management process is shown in accordance with an embodiment of the present specification.

[0033] Figure 7 An example diagram for illustrating relationships between address spaces in a trusted execution environment architecture is shown in accordance with an embodiment of the present specification.

[0034] Figure 8 An example flow diagram of a cross-VM communication process is shown in accordance with an embodiment of the present specification.

[0035] Figure 9 An example flow diagram of a cross-VM communication process is shown in accordance with an embodiment of the present specification.

[0036] Figures 10A-10F A benchmarking result example graph under micro-benchmarking is shown in accordance with an embodiment of the present specification.

[0037] Figure 11 Raw scores of various trusted execution environment architectures under macro-benchmarking are shown.

[0038] Figures 12A-12C A comparison graph of normalized scores of various trusted execution environment architectures under macro-benchmarking is shown.

[0039] Figure 13Example graphs comparing WOLFSSL performance on a Trusted Execution Environment architecture and Intel SGX according to embodiments of this specification are shown.

[0040] Figures 14A-14C A schematic diagram illustrating example performance test results of running a real-world application in a trusted execution environment architecture according to an embodiment of this specification is shown.

[0041] Figure 15 An example schematic diagram of a trusted execution environment architecture based on a computer system implementation according to an embodiment of this specification is shown. Detailed Implementation

[0042] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0043] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0044] Intel SGX, as a mainstream TEE platform, has become a de facto standard for building TEE-based applications. Intel SGX provides a Trusted Execution Environment (TEE) by isolating security-critical components of an application in a hardware-protected Enclave. In Intel SGX, everything outside the Enclave, including the operating system, is untrusted. The Trusted Computing Base (TCB) consists only of the underlying hardware and the Enclave itself. In this specification, the term "TEE" refers to a secure area running in the processor that runs in an isolated environment and in parallel with the operating system. The TEE protects data and code by using both hardware and software to ensure that the confidentiality and integrity of code and data loaded in the trusted execution environment are protected. The term "Enclave" refers to a secure area isolated in the trusted execution environment.

[0045] Intel SGX provides confidentiality and integrity guarantees for software running inside the Enclave. Regarding confidentiality, Intel SGX encrypts, protects, and physically isolates the memory of the Enclave in the Processor Reserved Memory (PRM), which is referred to as the Enclave Page Cache (EPC). Any software outside the Enclave cannot access the code and data in the EPC. In addition, a hardware memory encryption engine (MEE) is located between the processor and the memory controller, so that all memory accesses to the PRM region are encrypted and decrypted on the fly, i.e., on the fly encryption and decryption while performing the memory access. Regarding integrity, Intel SGX uses a Message Authentication Code (MAC) to encrypt and authenticate the EPC pages when they are swapped out from memory. Thus, Intel SGX can prevent direct tampering from outside software. Once decrypted and integrity checked, the EPC pages can be swapped in. In addition, Intel SGX also provides various platform support, including launch services and remote attestation services, so that the Enclave and SGX hardware can authenticate their own identity to a remote party.

[0046] However, the Instruction Set Architecture (ISA) extension of SGA explicitly classifies software applications into trusted and untrusted software applications, such that trusted software applications are isolated within a protected Enclave region that is only accessible by code executing in Enclave mode. Again, developers need to refactor existing applications or build new applications according to the SGX software specification. And the applications are compiled using the SGX SDK. As a result, applications developed for SGX can only run on SGX processors, leading to a vendor lock-in situation.

[0047] AMD SEV provides a VM-based TEE by encrypting Virtual Machines (VMs) without a trusted hypervisor (hereinafter, the hypervisor can also be referred to as a manager) or host (hereinafter, the hypervisor and host are used interchangeably when referring to a virtual machine monitor). Specifically, VM data in memory is protected using the AES engine in AMD System On Chip (SOC). When data of a SEV VM is written to memory, the data of the SEV VM is automatically encrypted via the AES engine, and when the data of the SEV VM is read from memory, the data of the SEV VM is automatically decrypted. Each SEV VM has a unique 128-bit VM Encryption Key (VEK). The VEK is stored in the AMD Secure Processor (AMD-SP) and is never exposed to the platform host. SEV-ES (SEV-Encrypted State) further protects the VM by encrypting the VM’s control block (VMCB) during VMEXIT. The latest version of SEV-SNP includes an inverted page table, such that the integrity of the encrypted memory and nested page tables is protected.

[0048] Similar to Intel SGX, everything outside of the SEV VM (including co-resident virtual machines as well as the platform host) is untrusted. Even though the hypervisor has access to the memory of the SEV VM, AES encryption prevents the hypervisor from compromising the confidentiality and integrity of the guest (customer) VM. Unlike Intel SGX, however, SEV supports page-level encryption with the entire Enclave being encrypted, and the guest VM can control which pages in memory are encrypted. The guest VM can modify encrypted pages to unencrypted pages by unsetting the C-bit in the customer page table entry (PTE) to share data with the hypervisor. Another prominent feature of AMD SEV is that the SEV VM does not require modification of application software, even though some modifications to the operating system kernel are necessary to enable SEV on both the customer and host sides. Since Linux kernel 4.16 and 5.10, there have been official patches to support SEV and SEV-ES. SEV also provides a remote attestation framework to protect the integrity and confidentiality of the VM during setup.

[0049] Unlike SGX, which provides an isolated and encrypted memory region within the address space of an application, the security boundary enforced by SEV is the entire VM's physical memory. SEV hardware does not provide isolation within a VM. Therefore, to protect the Enclave code and data from untrusted application code, the only secure and viable solution is to run the Enclave in a different SEV VM from the application and handle its instruction execution and communication properly.

[0050] In view of the above, embodiments of the present specification provide a trusted execution environment architecture that can provide binary code compatibility of partitioned SGX Enclave software and can be directly executed on AMD SEV. The trusted execution environment architecture can be viewed as an SGX hardware module that plugs into an SEV machine. In the trusted execution environment architecture, a trusted Enclave of an old SGX application is executed in a separate VM with VM protection provided by SEV. The separate VM is referred to as an Enclave VM (EVM), and the original VM that runs the untrusted part of the application is referred to as an application VM (AVM). Virtualization is achieved by plugging in SGX instructions (e.g., EENTER and EEXIT) during execution in the respective VM or across VMs, and implementing corresponding logic in the VM kernel to provide transparency to the Enclave code or the application code. Enclave code and data confidentiality is achieved using AMD’s memory encryption engine (MEE), while integrity is achieved through an attestation service that builds a chain of trust based on AMD’s SEV attestation. In accordance with the above-described scheme, when an SGX application is run in the trusted execution environment architecture, a comparable level of security to Intel SGX is achieved while preserving the unique security that AMD SEV provides for applications.

[0051] To enforce cross-Enclave isolation, only one Enclave is allowed to occupy the EVM, and the EVM is never reused. By using VM isolation and memory encryption of SEV, the application and hypervisor cannot access the Enclave memory, and different Enclaves are isolated, such that different Enclaves cannot access each other’s memory.

[0052] Furthermore, to transparently support Enclave binaries and applications that are originally built for Intel SGX processors, a cross-VM communication mechanism must be provided to facilitate SGX instruction emulation execution, cross-domain memory access, exception handling, etc.

[0053] In using AMD SEV to virtualize SGX Enclaves, there are many technical challenges. These technical challenges include, for example, (1) instruction emulation; (2) memory management; (3) Enclave entry and Enclave exit; (4) multi-Enclave and multi-threaded running; and (5) SGX remote attestation.

[0054] Although both Intel machines and AMD machines are x86-64 Instruction Set Architecture (ISA), the SGX extensions of x86-64 ISA are Intel specific and not supported for AMD machines. How to intercept and emulate these extended SGX instructions becomes a problem to be solved for the trusted execution environment architecture of the present specification.

[0055] Intel SGX embeds Enclave memory within the address space of an application program and allows Enclave code to access memory located inside and outside the Enclave, while prohibiting external (including another Enclave) access to Enclave memory. In the case of a multi-VM execution model, the trusted execution environment architecture of the present specification still needs to meet the same requirements as described above.

[0056] In addition, it is also necessary to preserve the semantics of Enclave Entrance (EENTER) and Enclave Exit (i.e. EEXIT) of Intel SGX in the trusted execution environment architecture of the present specification. Enclave Entrance refers to the world switch from untrusted space to trusted space, and Enclave Exit refers to the world switch from trusted space to untrusted space. The control flow of Enclave code must be preserved as in Intel SGX.

[0057] In the trusted execution environment architecture of the present specification, it is necessary to support running multiple Enclaves at the same time (for example, in addition to the Enclave program, such as a Quoting Enclave in SGX), and at the same time each Enclave should support multi-threading. In addition, in the trusted execution environment architecture of the present specification, it is also necessary to support SGX remote attestation.

[0058] The AMD SEV-based trusted execution environment architecture and trusted execution system according to embodiments of the present specification will be described below with reference to the accompanying drawings.

[0059] Figure 1 An example schematic diagram of an AMD SEV-based trusted execution system 1 according to embodiments of the present specification is shown.

[0060] As shown in Figure 1 , the trusted execution system 1 includes an Enclave Virtual Machine (EVM) 10, an Application Virtual Machine (AVM) 20, a hypervisor 30 and a bottom layer AMD SEV hardware architecture 40. In addition, the trusted execution system 1 also includes an AMD SEV-based trusted execution environment architecture. In Figure 1In the example of FIG. 1, the components of the trusted execution system 1 at the EVM 10 constitute a trusted computing base (TCB), while the components outside the EVM 10 are untrusted components. The trusted execution environment architecture includes an instruction emulator, an Enclave manager, a memory manager, and a cross-VM communicator. For simplicity of description, the trusted execution environment architecture according to embodiments of the present specification can also be referred to as trusted execution environment architecture 1 hereinafter.

[0061] The instruction emulator is configured to hook a #UD trap handler in the kernel of both the Enclave virtual machine and the application virtual machine to emulate execution of SGX instructions. As shown in Figure 1 The instruction emulator can include an instruction emulator 110 deployed in the EVM 10 and an instruction emulator 210 deployed in the AVM 20, as shown in Figure 2 The SGX instructions supported by the trusted execution environment architecture are listed in

[0062] The instruction emulator is configured to emulate execution of all ENCLS instructions on the boundary between the EVM 10 and the AVM 20. Figure 3 An example flowchart of the ENCLS instruction emulation execution process 300 according to embodiments of the present specification is shown in FIG. 3.

[0063] As shown in Figure 3 When the invalid opcode corresponds to an ENCLS instruction, at 310, the instruction emulator 210 in the AVM 20 packs the parameters of the ENCLS instruction and necessary data (if any, such as in the case of EADD) into a request data packet. Next, at 320, the instruction emulator 210 sends the request data packet to the instruction emulator 110 in the EVM 10 for execution.

[0064] In response to receiving the request data packet, at 330, the instruction emulation execution is completed in the EVM 10 with the parameters of the ENCLS instruction and necessary data in the request data packet. Then, at 340, the instruction emulator 110 in the EVM 10 returns the execution result as a response (a response data packet) to the instruction emulator 210 in the AVM 20.

[0065] Furthermore, when the instruction emulator simulates the execution of the ENCLS instruction, if the ENCLS instruction is not an EWB instruction, the returned execution result includes an error code and the register values ​​that need to be updated. If the ENCLS instruction is an EWB instruction, since the EWB instruction encrypts the Enclave page and writes it back to untrusted memory, the instruction emulator processes the EWB by packaging the encrypted Enclave page and its metadata into the response packet; that is, the returned execution result includes the encrypted Enclave page and its metadata. Alternatively, to reduce overhead, the response packet may not include the page payload and metadata if the EWB instruction simulation fails.

[0066] The instruction emulator is also configured to simulate the execution of ENCLU instructions, excluding the EENTER, EEXIT, and ERESUME instructions, within the Enclave virtual machine, according to hardware specifications (e.g., hardware information).

[0067] The emulation of most ENCLU instructions (such as EREPORT and EGETKEY) can be handled internally within the EVM10 according to the hardware specifications. However, the emulation of EENTER, EEXIT, and ERESUME instructions involves control flow transfer across the boundaries of two VMs. Therefore, the instruction emulator is also configured to emulate the execution of EENTER, EEXIT, and ERESUME instructions across VMs. Optionally, the EVM 10 and AVM 20 are designed to execute only one cross-VM instruction at a time.

[0068] Figure 4 An example flowchart illustrating cross-VM emulation of the EENTER, EEXIT, and ERESUME instructions according to an embodiment of this specification is shown.

[0069] like Figure 4 As shown, in the steps The #UD trap handler at AVM 20 intercepts the EENTER command, prepares the corresponding parameters, and performs the following steps: The process generates an EENTER request and sends it to the EENTER Handler in EVM 10, thereby passing the execution flow to the Enclave thread (i.e., the new thread) in EVM 10. In this specification, the Enclave thread refers to the execution thread of the Enclave application. The execution thread of the application at AVM 20 is then paused to wait for the corresponding EEXIT request (step 10). ). Upon receiving the EEXIT request by the Trap Handler at AVM 20, the EENTER Handler at AVM 20 sends an EENTER request to EVM 10 at step , the execution thread of the suspended application is resumed in AVM 10.

[0070] Upon receiving the EENTER request sent by the EENTER Handler at AVM 20, the #UD Trap Handler at EVM 10 sends an EENTER response to the EENTER Handler at AVM 20 at step , the EENTER Handler sends a notification to the manager to create an Enclave thread or pick up an existing Enclave thread (i.e., a new thread) to handle (step ). When the EEXIT instruction is executed (step ), the #UD Trap Handler at EVM 10 will intercept it, generate an EEXIT request and send the EEXIT request to the Trap Handler in AVM 20, thus passing the execution flow back to AVM 20 (step ), in addition, the #UD Trap Handler at EVM 10 can optionally terminate the execution of the new thread if necessary (step ). In addition, the EENTER Handler can also return an EENTER response to the #UD Trap Handler at AVM 10. In the example of Figure 4 , steps that can be performed in parallel are labeled with the same index and distinguished using solid and hollow circles.

[0071] In Intel SGX, an asynchronous Enclave eXit (AEX) can be triggered by an interrupt or an error. In the trusted execution environment architecture 1 of the present specification, only AEX triggered by an error (FAULT) is supported. Hardware interrupts in EVM 10 can be handled by the trusted Enclave kernel without causing AEX. Figure 5 An example flowchart of an asynchronous Enclave exit trigger process according to an embodiment of the present specification is shown.

[0072] As shown in Figure 5 , at step , the #UD Trap Handler in EVM 10 is hooked to check whether the error is triggered by an Enclave thread. If the error is triggered by an Enclave thread, at step The #UD Trap Handler will invoke the AEX Handler in the EVM 10, and then put the Enclave thread into sleep mode, similar to the case of EENTER on the AVM 20 side, so that there is no need to save the state of the Enclave using XSAVE as in Intel SGX. Next, at step , the AEX Handler will follow the AEX semantics of Intel SGX and generate a synthetic register state, and then send the synthetic register state to the #UD Trap Handler at the AVM 10. Subsequently, at step , the execution thread of the application in the AVM 20 is then woken up to invoke the corresponding error handler to perform error handling using the synthetic register state in the context of the #UD trap of the previous EENTER instruction. When the error handling is completed, the application crashes or the error has been properly handled, and at step , the control flow is transferred to the Asynchronous Exit Pointer (AEP) specified when the application executed the EENTER instruction, which points to the ERESUME flow to re-execute a new thread in the EVM 10 (steps to ). Specifically, the APP in the AVM 20 sends an ERESUME request to the #UD Trap Handler at the AVM 20 (step ). The #UD Trap Handler at the AVM 20 sends the ERESUME request to the ERESUME Handler at the EVM 10 (step ) for ERESUME request processing, and sends the processed ERESUME request to the #UD Trap Handler at the EVM 10. The #UD Trap Handler returns the processed ERESUME request to the new thread (step ), thereby re-executing the new thread.

[0073] Further, the #UD Trap Handler can wait to receive the ERESUME request when no processed ERESUME request is received from the ERESUME Handler (step ). Further, the ERESUME Handler at the EVM 10 can return an ERESUME response to the #UD Trap Handler at the AVM 20 after completing the ERESUME request processing (step ).

[0074] Further, in the examples of Figure 5 steps that can be performed in parallel are labeled with the same index and distinguished using solid and open circles.

[0075] Returning to Figure 1 , the Enclave Manager 120 is deployed in the EVM 10 and is configured to create and manage Enclaves. The Enclave Manager is a user space wrapper process for the host Enclave created inside the EVM 10. The Enclave binary is loaded as a shared library in the user space process. When the EVM 10 is started, the Enclave Manager is created and then suspended to wait for the creation of an Enclave. Since only one Enclave is allowed in the EVM 10, the Enclave Manager 120 creates only one Enclave (as shown in FIG. 1). In this specification, an Enclave is an execution environment composed of hardware and software, in which an Enclave application can be executed, and each Enclave program can include multiple Enclave threads.

[0076] Figure 6 An example flowchart of an Enclave application management process 600 according to an embodiment of the present specification is shown.

[0077] As shown in Figure 6 , at 610, an Enclave context dedicated to Enclave execution is created, and the created Enclave context is separated and isolated from the manager context of the Enclave Manager. The Enclave Manager will be created after a new EVM is started and suspended until the Enclave application is loaded. The Enclave Manager will create an isolated address space and corresponding threads for the Enclave application.

[0078] At 620, a memory synchronization thread and a distribution thread are configured and made to perform respective actions by trapping into the kernel code of the EVM 10 to enter the Enclave context.

[0079] At 630, the Enclave Manager registers itself as an idle Enclave Manager to the kernel of the EVM (Enclave Kernel) to wait for an EENTER request for creating an Enclave thread, such as ECREATE or ELDB / ELDU.

[0080] At 640, in response to receiving the EENTER request, an Enclave thread for executing the Enclave code is created. Then, at 650, after the Enclave thread is properly set up and detached from the manager thread, the context of the Enclave thread is switched to the Enclave context.

[0081] Optionally, in one example, to follow the Intel SGX semantics of disallowing Enclaves from accessing any system services by disabling instructions such as system call (SYSCALL), the kernel interface of the EVM 10 can be set up such that the Enclave thread is not allowed to use any system call instructions or software interrupts, and the Enclave context is not mapped to any system call instructions. As such, an interrupt will force the thread to be switched back to the manager context instead of handling the thread in the Enclave context.

[0082] The memory manager is configured to manage the internal memory of the EVM 10 to allow the Enclave application to access the internal memory of the EVM 10 as well as the external memory of the EVM 10, while disallowing the external components of the EVM 10 to access the internal memory of the EVM 10. As Figure 1 As shown, the memory manager includes a first memory manager 130 deployed in the kernel of the EVM 10 and a second memory manager 230 deployed in the kernel of the AVM 20.

[0083] In the trusted execution environment architecture 1 provided by the embodiments of the present specification, there are four address spaces: an Enclave page cache (EPC) address space, a virtual address space, an Enclave physical address space, and a manager address space. The manager address space is managed by the Enclave kernel, and the other three address spaces are used and managed by the trusted execution environment architecture.

[0084] Figure 7 An example schematic diagram for illustrating the relationship between the address spaces in the trusted execution environment architecture according to the embodiments of the present specification is shown. As Figure 7As shown, the application in AVM 20 and the Enclave in EVM 10 share the same virtual address space. EPC addresses are used as physical addresses in AVM 20, and as indices into the Enclave Page Cache Map (EPCM) in EVM 10, which is the EPC management architecture of Intel SGX. Enclave physical addresses are the real backing of Enclave memory that is mapped to virtual addresses in EVM 10, and are used by the CPU to perform addressing. Since Enclave applications can use the ENCLU instruction, which uses virtual addresses to negotiate with the EPCM, it is possible to create a Virtual-EPC mapping table inside the Enclave to help implement the ENCLU instruction. When using the ENCLS instruction to add or remove EPC pages, entries in the table are added or removed.

[0085] The memory manager is configured to manage the virtual address mapping and access permissions of each EPC page using a software EPC mapping table, each entry of which stores the virtual address mapping and access permissions of each EPC page.

[0086] Intel SGX securely manages the virtual address mapping and access permissions of EPC pages using the EPCM. To manage EPC pages, the trusted execution environment architecture according to embodiments of the present description implements a software EPCM. Similar to Intel SGX, each entry of the software EPCM stores the mapping and permissions of each EPC page.

[0087] Since SEV processors do not directly use the software EPCM, in the trusted execution environment architecture of embodiments of the present description, when an SGX instruction modifies an EPCM entry (such as the entry listed in Figure 2 By doing so, the restrictions of the EPCM can be reflected in regular memory accesses, such that the page tables in EVM 10 are isolated from AVM 20, making the page tables in EVM 10 trustworthy.

[0088] Some instructions like ECREATE and EPA can add pages without providing a virtual address. The Enclave Kernel allocates these pages in the kernel. ECREATE is Enclave-specific, so the page allocation added by ECREATE is performed in the corresponding EVM 10. EPA creates Version Array (VA) pages that are shared between Enclaves, but each entry in the VA is Enclave-specific. The Enclave Kernel creates the VA pages for each Enclave and encrypts them with Enclave-specific keys, then stores the encrypted VA pages in the AVM Kernel.

[0089] The memory manager is also configured to use a Fetch-and-Map mechanism that hooks into the page fault handler of the EVM 10 to enable Enclave code running in the EVM 10 to access memory in the AVM 20.

[0090] To allow code running in the EVM 10 to access memory in the AVM 20 that is out-of-enclave, a Fetch-and-Map mechanism that hooks into the page fault (#PF) handler of the EVM 10 is designed to fetch pages from the AVM 20. Specifically, when a page fault occurs, the #PF handler sends a request to the AVM 20 using the virtual address of the faulty page. If the page is mapped in the AVM 20, its data is sent back to the EVM 10 so that the #PF handler can map a new page initialized with the received data to the faulty address of the Enclave execution thread. This process is similar to the demand paging mechanism. The page is mapped as non-executable to ensure that out-of-enclave code cannot be executed in Enclave mode.

[0091] Additionally, the memory manager is optionally configured to not perform Fetch-and-Map on pages whose virtual addresses fall within the Enclave Linear Address Range (ELRANGE). If the virtual address of the page that caused the page fault falls in the ELRANGE, the memory manager will follow the AEX procedure and notify the AVM to handle the page fault.

[0092] The memory manager is also configured to use a Switchless Syncing mechanism to maintain page synchronization on both sides of the EVM 10 and the AVM 20.

[0093] When the trusted execution environment architecture maps pages between EVM 10 and AVM 20, it has to keep the pages synchronized. Inspired by the no-switch OCall, a similar no-switch synchronization mechanism is designed that uses a background worker thread to synchronize the mapped pages without switching in and out of the Enclave.

[0094] Specifically, both EVM 10 and AVM 20 set up a thread during initialization, which is called the no-switch synchronization worker thread. When a Fetch-and-Map event occurs, the address and content of the mis-mapped page are registered to the no-switch synchronization list on both sides. The no-switch synchronization worker thread monitors and synchronizes the changes of each page in the list periodically (e.g., every 100 milliseconds).

[0095] In addition, to avoid rewriting the changes of pages that have not been synchronized, a 4096-bit bitmap is used for page synchronization. Each bit in the bitmap corresponds to one byte of a page, where a 1-bit indicates that the corresponding byte has changed, and a 0-bit indicates that the byte has not changed. In this way, this bitmap helps to mask out all unchanged bytes, so that only those changed bytes will be synchronized.

[0096] Back to Figure 1 , the cross-VM communicator is configured to implement the cross-VM communication between EVM 10 and AVM 20.

[0097] In the trusted execution environment architecture 1 according to embodiments of the present specification, the cross-VM communication is used to provide services for instruction emulation and memory synchronization. In the present specification, the cross-VM communication can be implemented in multiple ways, for example, using an encrypted TCP connection. Alternatively, in one example, to achieve better performance, the cross-VM shared page can be used to transfer data. Such communication must involve the untrusted hypervisor in the threat model. Therefore, a secure cross-VM communication protocol needs to be designed.

[0098] The cross-VM communication protocol needs to satisfy the following properties.

[0099] • Arbitrary data size: By splitting a large chunk of data into smaller chunks and encapsulating them into data packets, the trusted execution environment architecture according to embodiments of the present specification can allow arbitrary size of data. The size of a data packet is the maximum data size that can be sent in each round of communication. The data packet has a fixed-size packet header that contains the total number of data packets, the index of the current data packet, and the total size of the data. The sender sends the data packets one by one, and the receiver splices them together to recover the entire data.

[0100] • Concurrency: Multiple senders can send data at the same time. The Enclave architecture according to embodiments of the present specification can ensure that they can send data at the same time without collision. To this end, a unique sequential session number is included in each data packet header. For large blocks of data, the data stream of these data packets is treated as a single session (e.g. call EADD). On the receiver side, the data packets of the corresponding session can be spliced to reconstruct the data.

[0101] • Confidentiality: Since the hypervisor is untrusted, the Enclave architecture according to embodiments of the present specification can ensure that the hypervisor cannot read the communication data. To achieve confidentiality, end-to-end symmetric encryption such as AES-GCM 128 can be used. The Enclave architecture according to embodiments of the present specification encrypts the entire data packet including the packet header to ensure that no data is leaked to the hypervisor.

[0102] • Integrity: A malicious hypervisor can alter data during transmission. The data must be ensured to reach the destination unmodified. To achieve integrity, the Enclave architecture according to embodiments of the present specification attaches a message authentication code (MAC) encrypted with a key to each data packet. Without the correct key, the hypervisor cannot modify the contents of the data packet.

[0103] • Multiple targets: Since there can be multiple EVMs, the Enclave architecture according to embodiments of the present specification needs to be able to send data packets to a specific VM. To this end, EVMs are assigned an EVM ID (e.g. a natural number indicating the order in which the EVMs register themselves to the hypervisor). The Enclave architecture according to embodiments of the present specification ensures that the encryption and verification keys in each EVM are different, so even if an EVM is compromised, the communication of other Enclaves remains secure.

[0104] • Resend prevention: A malicious hypervisor can resend outdated but legitimate data packets to an EVM. Therefore, the Enclave architecture according to embodiments of the present specification needs to ensure that all data that reaches the destination is new. Resend prevention has been achieved by using a unique session number for each data packet, which is also encrypted and integrity-protected.

[0105] • Secure key distribution: The Enclave architecture according to embodiments of the present specification assumes that a key is shared between the EVM and its AVM. This can be achieved securely, for example, by using a key that is pre-embedded in the AVM and the EVM, which can be configured by the user before deployment and protected from hypervisor attacks using image encryption. Other schemes for securely distributing keys can also be implemented.

[0106] Figure 8An example flowchart of the cross-VM communication protocol according to an embodiment of the present specification is shown, and Figure 9 An execution flowchart of the cross-VM communication protocol according to an embodiment of the present specification is shown.

[0107] As Figure 8 shown, the cross-VM communication protocol includes 10 steps, and its execution flowchart is shown as Figure 9 In Figure 8 and Figure 9 , each step is marked with a black circle number or a white circle number, where the black circle represents a step within the execution environment, and the white circle represents a step between execution environments.

[0108] The cross-VM communication protocol is described below using an example of data transmission from an untrusted component (AVM) to a trusted component (EVM). The process of data transmission from a trusted component (EVM) to an untrusted component (AVM) is similar to this process.

[0109] In step , the sender thread in the AVM 20 prepares data in shared memory, and then in step , signals the hypervisor using the CPUID instruction, the parameter of which indicates the ID of the target VM, as shown in FIG. 8. The hub 310 will pick up the data packet in the shared memory. The hub 310 is a component in the hypervisor 30 responsible for cross-VM communication. Next, the CPUID Handler requests (step ) the Send Worker to notify (step ) the target EVM via the IRQ Handler. The IRQHandler will use the CPUID to notify the hypervisor and take the data packet back to its shared memory (step ⑤). Then, the data packet is sent (steps to ) to the dispatcher (dispatcher) through the dispatch queue, in which the data inclusion is decrypted, spliced, and finally sent (step ) to the destination handler (Destination). The dispatcher is a kernel thread worker that dispatches data to its destination handler. The dispatcher and the destination handler are shown as two different modules in Figure 8 , but in implementation, the dispatcher can be integrated into the destination handler.

[0110] In addition, the trusted execution environment architecture 1 can also optionally include a remote authenticator (RAT) deployed in the underlying AMD SEV hardware structure and in the kernel of the Enclave virtual machine. Figure 1The remote attester is configured to remotely attest the Enclave application based on the remote attestation framework of AMD SEV (not shown). The remote attester is configured to remotely attest the Enclave application based on the remote attestation framework of AMD SEV (not shown).

[0111] The chain of trust of the trusted execution environment architecture 1 according to embodiments of the present specification relies on the remote verification framework of SEV to deploy EVM with the encrypted VM image of the trusted execution environment architecture provider, where any fused secret and mapped encryption key inside the EVM is only known to the provider.

[0112] The trusted execution environment architecture stores the hash of the root secret and the public key of the trusted execution environment architecture provider in the Enclave kernel, which is protected from attacks by the AVM 20 and the hypervisor. The root secret can be used to derive a private key using EGETKEY following the semantics of Intel SGX. The public key hash can enable the trusted execution environment architecture provider to launch a provider-signed Enclave with similar functionality, such as an Intel-signed Quoting Enclave. The EVM allows the provider-signed Enclave to derive a verification key in the Enclave kernel. Thus, the trusted execution environment architecture allows the Enclave code to perform remote verification using similar routines as Intel SGX applications. The attestation process of the Enclave uses the public key of the provider of the trusted execution environment architecture and the root secret stored in the Enclave kernel to simulate the key derivation instruction EGETKEY of Intel SGX to derive a verification key for remote attestation.

[0113] In embodiments according to the present specification, the trusted execution environment architecture extends the chain of trust of SEV to the Enclave virtual machine.

[0114] The root of trust for an AMD processor is a single pair of ARK private / public keys, Kark, known only to AMD's key distribution server (KDS). A pair of AMD SEV signing keys, Kask, also managed by the KDS, is used to generate a dedicated certificate for the SEV platform. The public key portion of Kask is signed with the private Kark. During manufacturing, each SEV platform is equipped with a pair of chip-unique EK (chip endorsement key) keys, Kcek, of which the public key portion of Keck is signed with Kask. During the initialization phase of the SEV platform, the SEV firmware generates a platform Diffie-Hellman key, Kpdh, and a platform EK key, Kpek. Kpek is signed with Kcek, and Kpdh is signed with Kpek. After initialization, the hypervisor retrieves the certificates for Kpdh and Kpek and the SEV firmware's unique platform ID. When the guest owner attempts to verify the platform (before starting a SEV VM on it), the hypervisor forwards these two certificates and the unique platform ID to the guest owner. The guest owner can then use the platform ID to obtain signed certificates for Kcek ID, Kask, and Kark from the KDS and then perform platform verification by verifying the following certificate chain: Kpdh Kpek Kcek ID Kask Kark.

[0115] To launch an EVM, the following steps need to be performed.

[0116] In step 1, the guest owner sends its DH public key, an encrypted Enclave image encrypted with a disk encryption key, Kblk, and an open virtual machine firmware (OVMF) file to the hypervisor.

[0117] In step 2, the hypervisor issues a LAUNCH START command, passing the guest owner's DH public key to the SEV firmware, thereby establishing a secure channel between the SEV firmware and the guest owner encrypted with a transport encryption key, Ktek, derived from DH.

[0118] In step 3, the hypervisor copies the OVMF file into memory and invokes a LAUNCH UPDATE DATA command to perform in-place memory encryption of the OVMF file.

[0119] At step 4, the hypervisor invokes the LAUNCH_MEASURE command to instruct the SEV firmware to compute a measurement of the OVMF memory, which is sent to the guest owner over a secure channel.

[0120] At step 5, the guest owner verifies the measurement and sends the Kblk encrypted using Ktek to the hypervisor. The hypervisor uses the LAUNCH_SECRET command to configure the encrypted Kblk into the launched guest VM, which is used by the OVMF to decrypt and load the encrypted image of the Enclave VM.

[0121] Thus, the following chain of trust is established: VM Kblk Ktek Kpdh.

[0122] With the ability to launch multiple Enclaves, the trusted execution environment architecture according to embodiments of the present description can support remote attestation for Intel SGX. A Quoting Enclave (QE) is used to provide remote attestation, the QE being an Enclave signed via the specific trusted execution environment architecture. The QE has a unique attribute signed via the specific trusted execution environment architecture that allows the QE to get the platform’s signing key Kp, which is derived by using a fusion key pre-deployed by the service provider of the trusted execution environment architecture 1. Kp is used to generate a signature that only the service provider of the trusted execution environment architecture 1 can verify. Both the QE and the trusted execution environment architecture image can be verified by confirming that the report is correctly signed with Kp.

[0123] The trusted execution environment architecture 1 according to embodiments of the present description supports the same remote attestation routine as Intel SGX.

[0124] At step 1, the remote party can trigger remote attestation by a challenge and the untrusted application needs the Enclave to be verified.

[0125] At step 2, the untrusted application now gets an ID bound to the platform (Intel side EPID) and sends it to the remote party.

[0126] At step 3, if the remote party accepts the ID, the Enclave performs EREPORT to generate a report containing the measurement and metadata of the Enclave with a digital signature using the derived signing key of the trusted execution environment architecture 1.

[0127] At step 4, the report is forwarded to the QE, which then checks if the report is signed correctly using the EREPORT key If the check passes, the report is signed using the platform's signing key The report is signed as quote and the quote is sent to the untrusted application.

[0128] At step 5, the untrusted application sends the quote to the remote party, who forwards the quote to the service provider of the Intel SGX (which acts as the Intel Attestation Service (IAS)). If the quote is approved by the service provider of the Intel SGX, the remote party then verifies the correctness of the measurement and metadata.

[0129] The chain of trust for the Enclave is established as follows: ( ) QE (VM).

[0130] It is noted that the remote party can use the DH key exchange algorithm to establish a secure channel with the Enclave during this process. The data of the DH key exchange can be signed by the QE and packed in the report. This ensures that the key exchange is secure and reliable.

[0131] The AMD SEV-based trusted execution environment architecture and trusted execution system according to embodiments of the present specification are described above with reference to Figures 1 to 9 The performance of the trusted execution environment architecture according to embodiments of the present specification will be discussed below.

[0132] Execution security

[0133] Regarding the ENCLS instruction and the ENCLU instruction

[0134] On Intel processors, the ENCLS instruction and the ENCLU instruction are implemented using microcode. The ENCLS instruction denotes a kernel privilege instruction, meaning an instruction that needs to be accessed through the kernel mode. Examples of ENCLS instructions include, but are not limited to, EADD, EBLOCK, ECREATE, EDBGRD, EDBGWR, EEXTEND, EINIT, ELDB / ELDU, EPA, EREMOVE, ETRACK, EWB, EAUG, EMODPR, and EMODT. The ENCLU instruction denotes a user privilege instruction, meaning an instruction that can be accessed in the user mode. Examples of ENCLU instructions include, but are not limited to, EENTER, EEXIT, EGETKEY, EREPORT, ERESUME, EACCEPT, EACCEPTCOPY, and EMODPE. The execution of these instructions is protected by the CPU hardware and cannot be intercepted by software programs. Although the trusted execution environment architecture 1 according to embodiments of the present specification cannot execute the ENCLS instruction and the ENCLU instruction using microcode, the trusted execution environment architecture 1 can implement the security of executing the ENCLS instruction and the ENCLU instruction using the following methods.

[0135] The ENCLS instruction is intended to be managed by the Enclave, so it is enough to include the parameters of the instruction in a request packet sent to the EVM to execute its function. End-to-end encryption is used when sending the request, so the hypervisor cannot modify the request packet sent. In addition, sanity checks specified in the SGX reference are also performed inside the EVM so that malicious requests will not be successful. The actual function of the instruction is executed in the EVM, so it cannot be intercepted by the AVM or software running in other EVMs.

[0136] The ENCLU instruction is mainly executed inside the EVM, except for the EENTER instruction, the EEXIT instruction, and the ERESUME instruction. The ENCLU instruction will be checked according to the SGX reference to ensure that they are secure. Since the ENCLU instruction is executed inside the EVM, its execution is trustworthy.

[0137] Regarding illegal instructions inside the Enclave

[0138] By not allowing Enclave code to execute instructions like SYSCALL, Enclave code is prohibited from accessing system resources in Intel SGX. The trusted execution environment architecture 1 can enforce this restriction by performing a check before the SYSCALL and software interrupt handler enters. If so, the handler will throw a #UD error. By doing so, it is ensured that Enclave code can never use system services in the EVM.

[0139] Enclave entry and Enclave exit

[0140] In the trusted execution environment architecture 1, the same Enclave entry and Enclave exit semantics as SGX are followed to transfer control flow to an Enclave that protects the security of execution.

[0141] For the EENTER instruction and the EEXIT instruction, the trusted execution environment architecture 1 allows control flow to be transferred into an Enclave via EENTER and out of an Enclave via EEXIT. Unlike SGX, the control flow transfer in the trusted execution environment architecture 1 crosses the boundary of two VMs. In the trusted execution environment architecture 1, EENTER puts the application thread to sleep and starts an Enclave thread in the EVM. Upon completion of the execution of the Enclave thread, EEXIT terminates the Enclave thread and wakes up the sleeping application thread. The only potential attack is to wake up the sleeping application thread prematurely and prevent future EEXIT, which can be performed by an attacker with kernel access to the AVM. However, this only affects the application in the AVM and not the EVM.

[0142] For AEX and ERESUME, similar to Intel SGX, control flow is transferred back to the AVM with synthetic state when an Enclave has an error, so that Enclave data including register state is never leaked. Unlike Intel SGX, which uses XSAVE to save the state of the Enclave, the trusted execution environment architecture 1 just puts the Enclave thread to sleep, so that it can be woken up and directly re-executed upon Resume. The Enclave thread sleeps within the EVM, so it is protected from being tampered by any attacker. However, unlike SGX, in the trusted execution environment architecture 1, AEX is not triggered by an interrupt or VMEXIT, but by a FAULT. This is because the EVM kernel is trusted to handle unrelated interrupts and exceptions, such as timer events, so unlike SGX, a context switch is not necessary.

[0143] Memory encryption and isolation

[0144] Regarding memory encryption

[0145] Both Intel® Architecture Code Integrity (IA CI) and SGX prevent software components outside the EVM (Enclave for SGX) from reading encrypted memory in plaintext and prevent physical attacks (e.g., cold boot attacks and DMA attacks) from directly reading secrets in encrypted memory. Although the memory encryption of SEV is not verified, it is slightly weaker than SGX, but SEV-SNP does protect memory integrity. Therefore, Intel® Architecture Code Integrity (IA CI) achieves comparable security to SGX.

[0146] In addition, SGX uses a single transient memory encryption key for all Enclaves, while SEV uses different keys for different VMs. Therefore, Intel® Architecture Code Integrity (IA CI) is more secure than SGX because it protects each Enclave in different VMs that are encrypted with different keys.

[0147] Regarding enforcing Enclave memory access rules and isolation

[0148] Intel® Architecture Code Integrity (IA CI) prevents access to Enclave memory if (a) the CPU is not in Enclave mode, (b) the corresponding EPCM entry is set to the block flag, (c) the target page is not a PT_REG page (i.e., a regular Enclave page), (d) the EID of the current Enclave is different from the owner of the page, and (e) the virtual address does not match the record of the EPCM entry. Intel® Architecture Code Integrity (IA CI) achieves a similar level of security guarantee via VM isolation. Intel® Architecture Code Integrity (IA CI) implements EPCM to maintain the metadata of each EPC page, including page type (e.g., PT_REG), virtual address mapping, access permission, etc.

[0149] While the EPCM maintained by the software is not consulted during page table walk, the EVM ensures that its page table correctly reflects the corresponding EPCM entries: (1) PT_REG pages (e.g., PT_SECS) all have user space mappings, so that they cannot be accessed by Enclave code; (2) when a page is transitioned to the blocked state, the trusted execution environment architecture sets its access permission to PROT_NONE, so that the page is inaccessible; (3) access permissions and virtual address mappings are correct. Thus, since the Enclave's page table is protected by the Enclave kernel in SEV VM, the trusted execution environment architecture 1 maintains the same level of security as SGX. Moreover, since the trusted execution environment architecture 1 implements one Enclave per VM and cannot reuse the EVM, it achieves similar Enclave isolation as Intel SGX, leveraging the VM isolation provided by SEV.

[0150] Restricted non-Enclave memory access in Enclave mode

[0151] Intel SGX allows code running in Enclave mode to access non-Enclave memory. However, it does not allow any non-Enclave memory to be mapped to virtual addresses within ELRANGE, which is reserved for Enclave memory. Furthermore, the entries TLB for non-Enclave memory pages loaded in Enclave mode are forced to set the non-executable (NX) flag, so as to ensure that the Enclave never executes code located outside the Enclave.

[0152] The trusted execution environment architecture 1 achieves the same level of security via Fetch-and-Map and non-switch synchronization. First, if the virtual address of non-Enclave memory falls into ELRANGE, the Fetch-and-Map never maps the Enclave memory to the EVM. Thus, any memory access to a virtual address in ELRANGE without a valid mapping directly triggers a page fault. Second, to prevent the execution of code in non-Enclave memory, the trusted execution environment architecture 1 also forces the fetched page to be non-executable. Furthermore, since the protected data in the Enclave must fall within ELRANGE, it is never fetched or synchronized with untrusted memory in the AVM, so that non-switch synchronization does not leak the protected data outside.

[0153] Cross-VM communication

[0154] The only interface exposed to the outside world by the EVM is the cross-VM communication interface. In the embodiment of the present specification, a data packet must fall into one of the following three categories: instruction emulation data packet, no-switch synchronization data packet, and Fetch-and-Map data packet.

[0155] The instruction emulation data packet is dispatched to the corresponding Enclave specified by the EPC page it operates on. The authentication is implemented in the local dispatcher of the Enclave according to the Intel SGX specification.

[0156] The no-switch synchronization data packet is first dispatched to its corresponding Enclave. Then, the Enclave will check its no-switch synchronization list to see if the page to be synchronized is in the list. Only when so is the data packet accepted. The correctness of the synchronization page is not an issue because the non-Enclave memory is not expected to be correct.

[0157] The Fetch-and-Map data packet will be compared with a list of threads waiting in the kernel, and see if one of the threads in the list is waiting for a specific address. If so, the data packet is accepted and the page is mapped to the non-Enclave memory in the EVM. If not, the data packet is discarded.

[0158] Therefore, since all data packets sent to the EVM are strictly examined, an attacker cannot send arbitrary data packets that are inconsistent with the SGX semantics. In addition, reordering of data packets does not introduce new security issues. For instruction emulation, since the trusted execution environment architecture 1 only allows one cross-VM instruction to be executed at a time, there is no need to worry about the hypervisor can reorder the execution. For memory-related data packets, reordering them can cause rewriting problems in untrusted memory. However, since the untrusted memory is not protected, such behavior does not introduce security issues in the SGX model.

[0159] Discussion on TCB size

[0160] In Intel SGX, the only software component inside the TCB is the Enclave binary. However, the microcode implementation of the SGX instructions is also part of the TCB because they are firmware running on hardware. In the Trusted Execution Environment Architecture 1, the TCB contains the Enclave kernel, the Enclave manager, and the Enclave binary. In the implementation of the present scheme, 8,840 lines of code were added to the Linux kernel 5.10.20. The Enclave manager is relatively small, with only 250 lines of code. So the overall TCB size change in the Trusted Execution Environment Architecture 1 is 9,090 lines of code plus the size of a minimal Linux kernel. Once a formally verified kernel such as seL4 is supported by AMD SEV-ES, the formally verified kernel can be used to replace the Linux kernel. This allows the entire additional components that have been added to the Enclave kernel to be fully trustworthy.

[0161] The Trusted Execution Environment Architecture 1 does not significantly increase the attack surface. Because the only interface exposed to the outside by the EVM is the cross-VM communication interface, the powerful attacks that an attacker outside the TCB can initiate are attacks against the cross-VM communication interface, such as eavesdropping, injection, dropping, and modifying communication packets. However, as discussed above, these attacks can be prevented by the verified encryption, replay prevention, and soundness checks performed on the EVM side. Therefore, the Trusted Execution Environment Architecture 1 can reduce the attack surface of the SEV VM to a level comparable to that of SGX.

[0162] The AMD SEV-based Trusted Execution Environment Architecture and the Trusted Execution System according to embodiments of the present specification are described as above and the system performance is discussed. Experiments will be employed below to evaluate the performance of the Trusted Execution Environment Architecture according to embodiments of the present specification.

[0163] In one experiment, the Trusted Execution Environment Architecture 1 according to embodiments of the present specification has been implemented with 16,167 lines of C language code (LoC) and 121 lines of x86-64 assembly language code. Among them, 6,377 lines of LoC and 121 lines of assembly language code are used to implement the AVM module, 8,840 lines of Loc are used to implement the Enclave kernel, 250 lines of LoC are used to implement the Enclave manager, and 700 lines of LoC are used to implement the hypervisor in the KVM module.

[0164] Performance overhead issues of the trusted execution environment architecture 1 will be discussed below. To this end, a set of benchmark programs and real-world applications are designed and selected to analyze the performance overhead at the component level and the application level. Among them, micro-benchmark programs are used to reveal performance instructions and component level, macro-benchmark programs are used to reflect the overall performance, and real-world SGX applications are used to reflect compatibility and performance overhead.

[0165] In this experiment, the SEV-ES platform is AMD's GIGABYTE MZ31-AR0, which is configured with an EPYC 7251 8-core processor with a running frequency of 2.1 GHz. The SEV-ES platform has 64 GiB of memory and is installed with Linux kernel 5.10.0 provided by AMD to support SEV-ES host capabilities. In this experiment, the virtual machine is configured to have 2 SMP cores and 4 GiB of memory, and each virtual machine has Linux kernel 5.10.20. In addition, a control experiment is also conducted on an Intel SGX machine, which is a DELL OptiPlex 5060 with an Intel Core i7-8700 6-core processor running at a frequency of 3.2 GHz, to compare the performance difference. The Intel SGX machine is equipped with 32 GiB of memory and runs Linux kernel 4.15.0. By default, all performance overheads are measured by running target benchmarks for 10 times, and then the average value is calculated.

[0166] A. Benchmark

[0167] Micro-benchmark

[0168] The trusted execution environment architecture 1 has multiple components responsible for executing Enclave programs. Enclave (1) needs to be initialized, (2) to execute ECall / OCall, (3) to execute specific SGX exit instructions, (4) to communicate across virtual machines, (5) to obtain Enclave external memory, and (6) to execute non-switch synchronization if necessary. Therefore, six micro-benchmark programs are designed to represent the performance and overhead related to these executions.

[0169] Enclave initialization

[0170] First, the overhead of creating and initializing an Enclave on the trusted execution environment architecture is measured, which usually involves a set of SGX instructions, such as ECREATE, EADD, EEXTEND, and EINIT. Specifically, Enclaves with different sizes (e.g., in the number of heap pages) are launched, and the results are as follows: Figure 10Athe block curve in Figure 9A (measurements for the trusted execution environment architecture 1). From Figure 10A It can be observed from

[0171] ECall / OCall latency

[0172] In essence, the ECall latency is the EENTER and EEXIT pair, and the OCall latency is the EEXIT and EENTER pair, so only one of them needs to be measured. To do so, the ECall latency is measured by implementing an empty ECall function and measuring how long it takes to execute the EENTER and EEXIT instructions. Note that this empty ECall function also includes code from the Intel SGX SDK. This empty ECall function is executed 200 times on Intel SGX and the trusted execution environment architecture 1, and the test results are shown in Figure 10B Figure 10B It can be seen from

[0173] SGX instruction latency

[0174] The SGX instruction latency is measured by running the SGX instruction 20 times, measuring the total time, and then calculating the average. This measurement is then repeated 10 times to estimate the average latency, and the measurement results are shown in Figure 10C ​ENCLS instruction involves 2, 3, or 4 packets. For ENCLS instructions, it takes approximately 0.9 milliseconds to complete a 2-packet instruction, approximately 1.3 milliseconds to complete a 3-packet instruction, and approximately 1.8 milliseconds to complete a 4-packet instruction. One exception is the ECREATE instruction, which takes about 3 milliseconds because ECREATE also sets up the Enclave management program for the new Enclave.

[0175] In addition to the EENTER instruction and ERESUME, the ENCLU instruction is typically executed within the Enclave. In the second micro-benchmark program, the performance of EENTER, ERESUME, and EEXIT used when the Enclave enters and exits the Enclave is measured, and thus EENTER, ERESUME, and EEXIT are placed empty. It can be seen that in most cases, the execution of these local ENCLU instructions can be completed within 5 microseconds. Note that EGETKEY and EREPORT are relatively slow because they involve cryptographic operations.

[0176] Cross-VM communication overhead

[0177] In the benchmark, the overhead of cross-VM communication is measured by recording timestamps before and after each step. Since the communication crosses the virtual machine boundary, the Precision Time Protocol (PTP) is used to synchronize the time of the VM and the hypervisor to sub-microsecond precision.

[0178] Figure 10D The measured delays are reported. Specifically, steps and are the CPUID event and the IRQ event, which take less than 10µs, and are not very important compared to other steps. For other steps, most of their delays are below 100µs, except for steps and step . This is because step is using a semaphore to pass data to the dispatch handler. Either scheduling or a busy scheduler can cause a long delay. The most time-consuming step is step , which handles the emulation of SGX instructions. The time variation of this step is very large because different emulation routines for different instructions can cause different overheads.

[0179] By adding up the delays of all steps, it can be seen that if the packet is the first packet of a 3-packet or 4-packet instruction, it does not involve step which takes about 300 µs. If the packet is part of an instruction that needs emulation, it takes about 600 µs. In summary, a 2-packet instruction takes about 900 µs, a 3-packet instruction takes about 1200 µs, and a 3-packet instruction takes about 1500 µs.

[0180] Memory acquisition overhead

[0181] When an Enclave accesses untrusted memory, there is a memory acquisition overhead. Therefore, a benchmark program was designed to measure the delay of accessing untrusted memory and compare it with the case where the page has already been acquired, and the result is shown in Figure 10E Accessing a local page or an acquired page takes about 0.4 µs, while acquiring a page causes a 0.9 µs delay. This result also shows that once a page is acquired, accessing it does not cause a large overhead.

[0182] Switchless synchronization overhead

[0183] The switchless synchronization delay was measured by measuring the round-trip delay and then dividing it by 2. Specifically, the EVM first modifies a non-Enclave page to trigger a switchless synchronization. When the AVM notices the page modification, it immediately changes it to trigger another synchronization. When that change is noticed by the EVM, a round-trip synchronization has been completed, and the measurement result is shown in Figure 10F From the figure, it can be seen that the average delay is about 53 µs.

[0184] Macro benchmark

[0185] NBENCH (also known as BYTEMARK) was run on the trusted execution environment architecture 1, a normal AMD SEV-ES VM, and an Intel SGX machine to compare their performance. Figure 11 The raw scores of the benchmark program running on the various platforms are shown. In addition, the raw scores were also normalized, and the normalized test results are presented in Figures 12A-12C

[0186] ​First, by comparing the Trusted Execution Environment Architecture 1 and the normal SEV-ES, it can be seen that the performance overhead (geometric mean) introduced by the Trusted Execution Environment Architecture 1 on the SEV-ES machine is 205%. In Figures 12A-12C, it can be seen that most of the tests show less than 3x slowdown, except for STRING SORT, BITFIELD, and LU DECOMPOSITION, which result in higher overhead. After examining the code, it was found that BITFIELD triggers a large number of ECalls and STRING SORT to randomly access large data objects in non-Enclave memory. The 6x slowdown for LU DECOMPOSITION is due to moderate ECalls. It can be seen that there are two factors that severely impact performance for the Trusted Execution Environment Architecture 1 : ECall frequency and non-Enclave memory access.

[0187] Second, the comparison of the Trusted Execution Environment Architecture 1 with Intel SGX shows a slowdown when migrating from SGX to the Trusted Execution Environment Architecture 1. The geometric mean of the overhead is 221%. The scores from a direct comparison between different CPU architectures can be used as a reference of how the application performs when migrating from an SGX machine.

[0188] Real-world SGX applications

[0189] Since cryptographic operations are typical operations for Enclave applications, and Graphene is complex enough to showcase the functionality of the Trusted Execution Environment Architecture 1, WOLFSSL and Graphene were run on the Trusted Execution Environment Architecture 1 to present the benchmark performance.

[0190] For the applications tested, most of them were run directly without any modification. The exception is those that use the CPUID instruction (e.g., Graphene), which must bypass the checking process.

[0191] WOLFSSL performance

[0192] The WOLFSSL performance was measured using a benchmark program named WOLFCRYPT. This benchmark program tests encryption, decryption, digest, and signature verification. The benchmark program was run on the Trusted Execution Environment Architecture 1 and SGX, and the results are shown in Figure 13 Figure 13 ​In the table, the ratio column represents the raw performance of the Intel SGX divided by the Trusted Execution Environment (TEE) architecture 1. It can be seen that for most encryption, decryption, and digest operations, the Intel SGX is approximately 0.5 times faster than the TEE architecture 1. For RSA algorithms and DH key exchange, the TEE architecture 1 even outperforms the Intel SGX. I / O-intensive signature verification and key generation are weaknesses of the TEE architecture 1. The geometric mean benchmark overhead indicates that the Intel SGX is approximately 0.9 times faster than the TEE architecture 1. Considering that the capabilities of the TEE architecture 1 are being tested in a virtualized environment using AMD's first-generation Zen server processors, which are less powerful than Intel desktop processors, this result is acceptable.

[0193] Graphene performance

[0194] We ran cURL, GMPbench, and Nginx on Graphene on Trusted Execution Environment Architecture 1 to test its ability to support large enclave applications. The startup time for a 256 MB Graphene was approximately 5 minutes on Trusted Execution Environment Architecture 1, and approximately 0.5 seconds on an Intel SGX. This is because starting such a large enclave requires a significant number of EADD and EEXTEND operations.

[0195] The time consumed by each specific instruction when starting Graphene is as follows: Figure 14A As shown. From Figure 14A As can be seen, EEXTEND causes 3 / 4 of the overhead because each EEXTEND can only hash 256 bytes, so it requires 16 EEXTENDs to hash the entire 4096-byte page.

[0196] To measure the performance of applications following Graphene, cURL and GMPbench were evaluated. The former is an I / O (networking) intensive workload, and the latter is CPU-bound. Grapehen-SGX on Trusted Execution Environment Architecture 1 was compared to Graphene-Direct mode, which runs the library OS directly outside the Enclave. For the cURL test, it was used to access a URL and latency was measured; its performance is shown in Figure 14B. Graphene Direct is approximately 7 times faster than Graphene-SGX on Trusted Execution Environment Architecture 1. This is because network traffic is handled outside the Enclave, resulting in a large number of OCalls and accesses to untrusted memory. Furthermore, the Enclave must copy those caches to its own memory, leading to additional accesses to untrusted memory. The GMPbench test results are as follows...Figure 14C As can be seen from Figure 14C The results show that the trusted execution environment architecture 1 does not increase the burden of CPU computation. These results show that the trusted execution environment architecture 1 is more suitable for CPU-intensive workloads such as key operations.

[0197] As described above with reference to Figures 1 to 14C , the AMD SEV-based trusted execution environment architecture and trusted execution system according to the embodiments of the present specification are described. The above trusted execution environment architecture can be implemented in hardware, or implemented in software or a combination of hardware and software.

[0198] Figure 15 A schematic diagram of a computer system-implemented trusted execution environment architecture 1500 according to an embodiment of the present specification is shown. As Figure 15 shown, the trusted execution environment architecture 1500 can include at least one processor 1510, a memory (e.g., a non-volatile memory) 1520, an internal memory 1530, and a communication interface 1540, and the at least one processor 1510, the memory 1520, the internal memory 1530, and the communication interface 1540 are connected together via a bus 1560. The at least one processor 1510 executes at least one computer-readable instruction (i.e., the above-mentioned elements implemented in software) stored or encoded in the memory.

[0199] In one embodiment, computer-executable instructions stored in the memory, when executed, cause the at least one processor 1510 to: hook an illegal opcode trap handler in the kernel of both the Enclave virtual machine and the application virtual machine to emulate execution of SGX instructions; create and manage Enclave applications; manage internal memory of the Enclave virtual machine to allow the Enclave applications to access the internal memory of the Enclave virtual machine and external memory of the Enclave virtual machine, while prohibiting external components of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and implement cross-virtual machine communication between the Enclave virtual machine and the application virtual machine.

[0200] It should be understood that the computer-executable instructions stored in the memory, when executed, cause the at least one processor 1510 to perform various operations and functions described above in the various embodiments of the present specification in conjunction with Figures 1-14C the above-mentioned elements implemented in software.

[0201] According to one embodiment, a program product such as a machine-readable medium (e.g., a non-transitory machine-readable medium) is provided. The machine-readable medium can have instructions (i.e., the above-mentioned elements implemented in software) that, when executed by a machine, cause the machine to perform the various operations and functions described above in the various embodiments of the present specification in conjunction withFigures 1-14C The various operations and functions described. Specifically, a system or apparatus equipped with a readable storage medium on which a software program code implementing the functions of any of the above-described embodiments is stored can be provided, and a computer or processor of the system or apparatus is caused to read and execute the instructions stored in the readable storage medium.

[0202] In this case, the program code read from the readable medium itself can implement the functions of any of the above-described embodiments, and thus the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present application.

[0203] Embodiments of the readable storage medium include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, a DVD- RW, a DVD-RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer or the cloud over a communication network.

[0204] According to one embodiment, a computer program product is provided, which includes a computer program that, when executed by a processor, causes the processor to perform the above-described various operations and functions in the various embodiments of the present specification. Figures 1-14C The various operations and functions described.

[0205] Those skilled in the art will understand that the various embodiments disclosed above can be variously modified and changed without departing from the spirit of the application. Therefore, the scope of protection of the present application should be defined by the appended claims.

[0206] It should be noted that not all steps and units in the above-described flowcharts and system block diagrams are necessary, and some steps or units can be omitted according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in each of the above embodiments can be a physical structure or a logical structure, i.e., some units can be implemented by the same physical entity, or some units can be implemented by multiple physical entities, or some units can be implemented by some components in multiple independent devices.

[0207] In the above embodiments, a hardware unit or module can be implemented by mechanical means, or by electronic means, or any combination thereof. For example, a hardware unit, module or processor can include an appropriately programmed processor, or a special purpose computer, or an ASIC programmed to perform a certain operation. A hardware unit or processor can also include a combination of permanent and programmable logic, e.g., a combination of a special purpose computer and a general purpose computer, or an ASIC and a general purpose computer. The specific combinations of hardware and software are not limiting and are chosen by a designer based on cost and time considerations.

[0208] The detailed description set forth above describes exemplary embodiments and does not represent all of the only embodiments that can be practiced under the claims. The term "exemplary" used throughout this description means "serving as an example, instance, or illustration," and not "preferred" over other embodiments. The detailed description includes specific details for the purpose of providing a thorough understanding of the described techniques. These techniques, however, can be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments.

[0209] The foregoing description of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the disclosure be limited not with this detailed description, but rather by the claims appended hereto.

Claims

1. An apparatus for providing a trusted execution environment based on AMD SEV, comprising: An instruction emulator hooks illegal opcode trap handlers into the kernels of both the Enclave virtual machine and the application virtual machine to simulate the execution of SGX instructions; Enclave Manager, deployed within the Enclave virtual machine, is used to create and manage Enclave applications; A memory manager manages the internal memory of the Enclave virtual machine, allowing Enclave applications to access both the internal and external memory of the Enclave virtual machine, while prohibiting external components of the Enclave virtual machine from accessing its internal memory; and A cross-virtual machine communicator enables cross-virtual machine communication between the Enclave virtual machine and the application virtual machine. The instruction simulator is configured as follows: Simulate the execution of all ENCLS instructions at the boundary between the Enclave virtual machine and the application virtual machine; According to the hardware specifications, the execution of ENCLU instructions, excluding the EENTER, EEXIT, and ERESUME instructions, is simulated within the Enclave virtual machine; and The EENTER, EEXIT, and ERESUME instructions are executed by emulation across virtual machines.

2. The apparatus of claim 1, further comprising: The remote authenticator, deployed in the underlying AMD SEV hardware architecture and the kernel of the Enclave virtual machine, is used to remotely authenticate the Enclave application based on the AMD SEV remote authentication framework.

3. The apparatus of claim 1, wherein, The instruction simulator is configured to simulate the execution of ENCLS instructions in the following manner: The parameters of the ENCLS instruction and the data from the instruction simulation execution are packaged into a request data packet and sent to the Enclave virtual machine for execution. as well as After the Enclave virtual machine completes the instruction simulation execution, it returns the execution result as a response to the application virtual machine.

4. The apparatus of claim 3, wherein, When the ENCLS instruction is not an EWB instruction, the execution result includes an error code and the register value that needs to be updated; when the ENCLS instruction is an EWB instruction, the execution result includes an encrypted Enclave page and its metadata.

5. The apparatus of claim 4, wherein, When the ENCLS instruction is an EWB instruction and the EWB instruction fails to execute, the response does not contain page payload and metadata.

6. The apparatus of claim 1, wherein, The instruction simulator is configured to simulate the execution of the EENTER instruction, the EEXIT instruction, and the ERESUME instruction in the following manner: The EENTER instruction is intercepted by the first illegal opcode trap handler in the kernel of the application virtual machine, the corresponding EENTER instruction parameters are prepared, an EENTER request is generated and sent to the EENTER handler in the Enclave virtual machine. The EENTER request includes the EENTER instruction parameters. The execution thread of the application in the application virtual machine is suspended to wait for the corresponding EEXIT. Upon receiving the EENTER request, the corresponding Enclave thread is obtained and processed via the Enclave hypervisor in the Enclave virtual machine. When the EEXIT instruction is executed in the Enclave virtual machine, the EEXIT instruction is intercepted by a second illegal opcode trap handler hooked into the kernel of the Enclave virtual machine, an EEXIT request is generated, and the EEXIT request is sent to the first illegal opcode trap handler in the application virtual machine. Upon receiving the EEXIT request, the execution thread of the application is resumed via the EEXIT handler at the application virtual machine.

7. The apparatus of claim 6, wherein, The instruction emulator is also configured to return the processing result of the EENTER request as an EENTER response to the first illegal opcode trap handler at the application virtual machine via the EENTER handler.

8. The apparatus of claim 6, wherein, The instruction emulator is also configured to terminate the execution of the Enclave thread in response to the second illegal opcode trap handler intercepting the EEXIT instruction.

9. The apparatus of claim 1, wherein, The instruction emulator is configured to exit asynchronously via an error-triggered enclave in the following manner: The error is checked to see if it was triggered by the Enclave thread via a second illegal opcode trap handler hooked into the kernel of the Enclave virtual machine; In response to the error being triggered by the Enclave thread, the asynchronous Enclave exit handler is called via the second illegal opcode trap handler, and the Enclave thread is put into sleep mode; The synthesized register state is generated via the asynchronous Enclave exit handler, and the synthesized register state is sent to the first illegal opcode trap handler at the application virtual machine; The execution thread of the application at the application virtual machine is awakened by the first illegal opcode trap handler, so that the corresponding error handler is called using the synthesized register state to perform error handling within the context of the illegal opcode trap of the previous EENTER instruction. In response to the completion of error handling, the execution thread of the Enclave application at the Enclave virtual machine is re-executed via the first illegal opcode trap handler, the re-execution handler, and the second illegal opcode trap handler, using the thread re-execution process pointed to by the asynchronous exit pointer.

10. The apparatus of claim 1, wherein, The Enclave manager is configured as follows: Create an Enclave context dedicated to Enclave execution, which is separate and isolated from the manager context of the Enclave manager; Configure a memory synchronization thread and a dispatch thread, and make the memory synchronization thread and the dispatch thread enter the Enclave context to perform actions by trapping into the kernel code of the Enclave virtual machine; Register itself with the kernel as an idle Enclave manager to wait for an EENTER request to create an Enclave thread; In response to receiving an EENTER request, an Enclave thread is created to execute the Enclave code; as well as After the Enclave thread is correctly set up and detached from the manager thread, switch the context of the Enclave thread to the Enclave context.

11. The apparatus of claim 10, wherein, The kernel interface of the Enclave virtual machine is configured such that the Enclave thread cannot use any system call instructions or software interrupts, and the Enclave context is not mapped to any system call instructions.

12. The apparatus of claim 1, wherein, The memory manager includes a first memory manager deployed in the kernel of the Enclave virtual machine and a second memory manager deployed in the kernel of the application virtual machine. The memory manager is configured as follows: The virtual address mapping and access permissions for each EPC page are managed using a software EPC mapping table, where each entry in the software EPC mapping table stores the virtual address mapping and access permissions for each EPC page. A retrieval and mapping mechanism, hooked to the page fault handler of the Enclave virtual machine, enables Enclave code running in the Enclave virtual machine to access memory in the application virtual machine; as well as A handover-free synchronization mechanism is used to maintain page synchronization between the Enclave virtual machine and the application virtual machine.

13. The apparatus of claim 12, wherein, The memory manager is configured to prohibit fetching and mapping operations on Enclave pages whose virtual addresses fall within the Enclave linear address range.

14. The apparatus of claim 12, wherein, The device's address space includes the EPC address space, the virtual address space, the Enclave physical address space, and the manager address space. The manager address space is managed by the kernel of the Enclave virtual machine. The virtual address space is shared by the application in the application virtual machine and the Enclave thread in the Enclave virtual machine. The EPC address is used as a physical address in the application virtual machine and as an index for the EPCM in the Enclave virtual machine. The Enclave physical address is a backup of the Enclave memory mapped to the virtual address in the Enclave virtual machine and is used to perform addressing.

15. The apparatus of claim 1, wherein, The cross-virtual machine communicator includes a first cross-virtual machine communicator deployed in the kernel of the Enclave virtual machine, a second cross-virtual machine communicator deployed in the kernel of the application virtual machine, and a hub deployed in the hypervisor. The cross-virtual machine communicator is configured to perform cross-virtual machine communication between the Enclave virtual machine and the application virtual machine using the following cross-virtual machine communication protocol: The sender thread in the first virtual machine prepares the data packet to be transmitted in shared memory and uses the CPUID instruction to signal the CPUID handler in the hub. The parameter of the CPUID instruction indicates the virtual machine ID of the target second virtual machine. The CPUID handler picks up data packets in the shared memory and requests the send worker thread in the hub to notify the target second virtual machine via the IRQ handler. The IRQ handler uses the CPUID instruction to notify the management program and retrieves the data packet back to the shared memory of the target second virtual machine; The data packet is sent to the distributor of the target second virtual machine through the distribution queue of the target second virtual machine; as well as The data is decrypted and concatenated via the distributor, and the concatenated data table is sent to the destination processing program at the target second virtual machine.

16. The apparatus of claim 2, wherein, The remote authenticator uses the public key provided by the Trusted Execution Environment (TEX) provider and the root key stored in the kernel of the Enclave virtual machine to simulate the SGX key retrieval instruction to obtain the verification key and achieve remote authentication.

17. A trusted execution system, comprising: The apparatus for providing a trusted execution environment based on AMD SEV as described in any one of claims 1 to 16; Enclave virtual machine; Application virtual machine; Management procedures; as well as The underlying AMD SEV hardware architecture.

18. A computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform: The illegal opcode trap handler is hooked into the kernels of both the Enclave virtual machine and the application virtual machine to simulate the execution of SGX instructions; Create and manage Enclave applications; Manage the internal memory of the Enclave virtual machine to allow Enclave applications to access both the internal and external memory of the Enclave virtual machine, while prohibiting external components of the Enclave virtual machine from accessing its internal memory; and To enable cross-virtual machine communication between the Enclave virtual machine and the application virtual machine, in, The method of hooking the illegal opcode trap handler into the kernels of both the Enclave virtual machine and the application virtual machine to simulate the execution of SGX instructions includes: Simulate the execution of all ENCLS instructions at the boundary between the Enclave virtual machine and the application virtual machine; According to the hardware specifications, the execution of ENCLU instructions, excluding the EENTER, EEXIT, and ERESUME instructions, is simulated within the Enclave virtual machine; and The EENTER, EEXIT, and ERESUME instructions are executed by emulation across virtual machines.

19. A computer program product comprising a computer program, said computer program being executed by a processor to implement: The illegal opcode trap handler is hooked into the kernels of both the Enclave virtual machine and the application virtual machine to simulate the execution of SGX instructions; Create and manage Enclave applications; Manage the internal memory of the Enclave virtual machine to allow Enclave applications to access both the internal and external memory of the Enclave virtual machine, while prohibiting external components of the Enclave virtual machine from accessing its internal memory; and To enable cross-virtual machine communication between the Enclave virtual machine and the application virtual machine, in, The method of hooking the illegal opcode trap handler into the kernels of both the Enclave virtual machine and the application virtual machine to simulate the execution of SGX instructions includes: Simulate the execution of all ENCLS instructions at the boundary between the Enclave virtual machine and the application virtual machine; According to the hardware specifications, the execution of ENCLU instructions, excluding the EENTER, EEXIT, and ERESUME instructions, is simulated within the Enclave virtual machine; and The EENTER, EEXIT, and ERESUME instructions are executed by emulation across virtual machines.

Images