High-performance heterogeneous secure memory

Abstract

The present invention provides a high-performance heterogeneous secure memory, including: an HSMEM transmission engine, a multi-mode protection engine and an interface; the HSMEM transmission engine establishes an HSMEM transmission channel to achieve high-speed transmission between the CPU and the GPU, the HSMEM transmission channel is located inside the chip, and controls the DMA request for the secure memory; each memory protection scheme supported by the multi-mode protection engine is called a mode, and a memory block is protected by different modes, including a mode selection module for selecting a mode for the memory block, a data encryption module for data encryption and decryption, and an integrity tree module for maintaining integrity trees of different modes; through the interface, developers use preset instructions to explicitly change the mode of the memory block. The present invention realizes high-performance data transmission between heterogeneous secure memories, spends less time on CPU-GPU transmission, and makes the overall performance of the application higher.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a high-performance heterogeneous secure memory. Background Art

[0002] After several years of development, artificial intelligence technology has been widely used in many fields, such as autonomous driving, face recognition, and medical imaging. The massive amount of training data and high-value artificial intelligence models have made data privacy issues a focus of attention. Due to their extensive resource requirements, modern artificial intelligence applications are usually deployed in high-performance data centers, such as where ChatGPT is trained, using more than 10K NVIDIA A100 GPUs. The complexity of the software stack and potential vulnerabilities (such as the Linux kernel with more than 360 million lines of code and more than 3K vulnerabilities), as well as potential malicious employees, threaten the data privacy of artificial intelligence applications. On the one hand, the platform administrator of the data center itself may have malicious behavior. On the other hand, the existing platform usually has a very deep software stack, which brings a large number of software vulnerabilities and attack surfaces. Once the cloud platform software system is hijacked, the security of the machine learning application cannot be guaranteed. Even if the training is carried out in the machine within the enterprise, the training process may be damaged by internal malicious employees or attackers. Therefore, how to ensure the security of machine learning applications and prevent them from being attacked by the hijacked operating system has become a problem that has received widespread attention from academia and industry.

[0003] Trusted Execution Environment (TEE) is widely used to protect applications from attacks by untrusted system software. TEE ensures that any component outside the TEE cannot access or modify the data flow and control flow of components inside the TEE. There are already many ways to build TEE, and almost all CPU manufacturers, including Intel, AMD, and ARM, have introduced their hardware TEE extensions. However, modern artificial intelligence applications rely heavily on heterogeneous accelerators, such as GPUs, NPUs, etc., which are widely used to accelerate the calculation of artificial intelligence applications. However, these accelerators cannot be directly protected by the TEE on the CPU side. Therefore, the security improvement of the TEE on the CPU side for artificial intelligence applications is very limited. Therefore, in order to protect the data security of artificial intelligence applications, it is necessary to build a trusted execution environment on the artificial intelligence accelerator to ensure that malicious operating systems cannot access the accelerator's data and tamper with the accelerator's computing process. GPU is a common accelerator for artificial intelligence applications, and building GPU TEE is an effective way to protect artificial intelligence applications.

[0004] Although high-bandwidth memory (HBM) is considered secure and not vulnerable to physical attacks, it is expensive, has limited production capacity, and is difficult to replace when it fails. Therefore, there will still be accelerators using GDDR memory for some time to come. Even in August 2023, when GPUs equipped with HBM like H100 have been on the market for a long time, NVIDIA announced the L40S GPU, a data center GPU product, but still uses GDDR memory. Only when the cost and production capacity of HBM are controllable in the future will GDDR have the possibility to complete its historical mission. In addition, most existing accelerator models, including various GPUs, still use GDDR memory. At the same time, HBM cannot solve the RowHammer and RowPress problems. Therefore, many academic studies are based on building accelerators with secure memory using traditional DDR or GDDR.

[0005] Accelerators, including GPUs, differ from CPUs in terms of secure memory. They can differ in two aspects: 1) The integrity tree structure may be different. 2) The encryption algorithm may be different. GPUs have multiple memory controllers, and in recent GPU secure memory designs, each memory partition has its own secure memory controller. By adopting multiple memory partitions, the parallel access capability of memory can be improved. Each memory controller uses local addresses within the partition and counters for AES encryption. In addition, some multi-chip module GPUs contain multiple GPUs and memory chips, which are connected through inter-chip connections with lower bandwidth than intra-chip connections. If a centralized engine is used for all memory partitions, the performance may not be ideal due to the intra-chip or inter-chip interconnects between the engine and the memory controller. Therefore, recent GPU secure memory architectures adopt a protection engine inside each controller, and each memory partition maintains its own integrity tree. In addition, some GPU secure memories use different encryption algorithms, such as AES-XTS. In addition, CPUs and GPUs differ in the granularity of memory protection. GPU cache lines are usually 128 bytes, while CPU cache lines are 64 bytes.

[0006] Trusted Execution Environment (TEE) is used to protect user data and programs. It can construct an isolation area that is logically isolated from the outside. However, logical isolation alone is not enough to defend against physical attacks. Hardware vendors have introduced memory protection engines in memory controllers to protect data from physical attacks. The strongest protection mechanism in current products can ensure confidentiality, integrity, and freshness, such as Intel SGX, which has an architecture similar to BMT (Bonsai Merkle Tree). A counter-based integrity tree is used to ensure that the correct counter is used. Memory blocks can be encrypted using counter values, keys stored on the chip, and memory addresses. In addition, a message authentication code (MAC) is calculated and stored to ensure the integrity of the block. The root of the integrity tree is stored on the chip and has never been tampered with. When a block is loaded into the chip, the integrity tree is used to calculate the correct counter and extract the MAC to verify the integrity of the block. When a dirty block is eliminated from the cache and written back to the memory, the counter value increases. In addition, there are different memory addressing methods, such as Figure 1 As shown. Assume there are k memory partitions and the interleaving granularity is g memory blocks. Even for this randomly interleaved memory, it still ensures that the consecutive k*g memory blocks come from different memory partitions. In the basic method of memory protection, the integrity of the counter is mainly protected by an integrity tree, and then a counter is used to encrypt a memory block, such as Figure 3 shown.

[0007] NVIDIA H100 GPU contains many security features that can limit untrusted software's access to GPU content. There is an on-chip security processor inside the GPU, which supports multiple types and levels of encryption and provides hardware-protected memory areas. The confidential computing function of NVIDIA H100 GPU needs to be used in conjunction with confidential virtual machines. When a confidential virtual machine is protected by TEE on the CPU side, a confidential virtual machine can be associated with several H100 GPUs with secure computing mode turned on, or several MIG computing instances. After that, the GPU will establish a secure channel mechanism with the confidential virtual machine on the CPU. The data transmission between the GPU and the CPU will be encrypted and decrypted by hardware. At the same time, the relevant hardware can ensure that untrusted virtual machine monitors cannot directly access the GPU in secure computing mode. NVIDIA H100 GPU allows users to verify that they are communicating with NVIDIA GPUs with confidential computing enabled and verify the security status of the GPU. However, the technology has a low transmission rate. According to its technical report, the CPU-GPU transmission speed is only 4GB / s, and only HBM memory is protected at the same time. GPUs that use a large amount of traditional memory cannot be protected.

[0008] MMT was published at the HPCA conference in 2023 (Efficient Distributed Secure Memory with Migratable Merkle Tree). In order to achieve (almost) linear secure data transfer speed between distributed enclaves, this work expands the scale of the memory protection engine from one node to multiple nodes. The memory protection engine is already able to protect the confidentiality, integrity and freshness of physical memory. By reusing hardware protection mechanisms, enclaves can directly transfer data to another enclave through various connections (such as PCIe, RDMA, etc.) without additional cryptography-based operations. This scheme proposes a migratable Merkle tree (MMT), which can transfer data and metadata to remote nodes without software participation (such as re-encryption). A key insight of MMT is that (single-node) hardware memory protection already provides confidentiality, integrity and freshness guarantees for untrusted DRAM, which can be reused for protection in untrusted networks. To this end, this work first expands a single integrity tree to an integrity forest across multiple nodes and breaks the limitation of encrypted metadata limited to the CPU. In addition, this work designs a new protocol: MMT closure delegation, which is used to protect data transmitted in untrusted networks. It can securely transmit the root, nodes, and data of the integrity subtree to remote nodes to prevent the leakage of confidential information or replay attacks. Finally, this work implements a tiny and trusted module to manage enclaves and trusted hardware modules. It hides the details of the hardware implementation and is responsible for the connection between local and remote enclaves. However, this technology can only support transmission between nodes with the same secure memory architecture. In heterogeneous scenarios, the architecture of the secure memory of the CPU and GPU is different, and this solution cannot be used for transmission. Summary of the invention

[0009] In view of the defects in the prior art, an object of the present invention is to provide a high-performance heterogeneous secure memory.

[0010] The high-performance heterogeneous secure memory provided by the present invention includes: an HSMEM transmission engine, a multi-mode protection engine and an interface;

[0011] The HSMEM transmission engine establishes an HSMEM transmission channel to achieve high-speed transmission between the CPU and the GPU. The HSMEM transmission channel is located inside the chip and controls DMA requests for secure memory. It selects metadata related to the memory to be transferred and transmits data without additional data encryption and decryption. For each data transfer, a preset number of cache blocks are encrypted. After the data is transferred to the CPU or GPU, the multi-mode protection engine is located in the memory controller and directly uses the original protection scheme to transmit data;

[0012] Each memory protection scheme supported by the multi-mode protection engine is called a mode. A memory block is protected by different modes. Inside the multi-mode protection engine, there is a mode selection module for selecting a mode for a memory block, a data encryption module for data encryption and decryption, and an integrity tree module for maintaining integrity trees of different modes.

[0013] Through the interface, developers use preset instructions to explicitly change the mode of a memory block.

[0014] Preferably, at initialization, the HSMEM transmission engine generates a key with the peer device and builds a traditional secure channel. When data is to be transmitted, the HSMEM transmission engine prepares metadata that needs to be transmitted with the data. In addition, the secure channel is used to exchange some basic system configurations during the initialization process.

[0015] The HSMEM transfer engine calculates the target address of the metadata corresponding to the data, and then the HSMEM transfer engine controls the DMA engine to copy the metadata to the target location. The GPU uses its own DMA engine for data transfer.

[0016] Preferably, during initialization, the HSMEM transmission engine includes an initialization module, specifically:

[0017] When the system boots up or the GPU is plugged in, the CPU and GPU authenticate each other and establish a traditional secure channel. Through the secure channel, the CPU and GPU negotiate multiple shared keys for memory encryption. In addition, the CPU and GPU provide each other with memory layout information, including the integrity tree format, encryption algorithm type, and the address of the metadata area. The GPU also provides the CPU with the number of memory controllers and the memory addressing scheme. After completing this operation, the CPU and GPU will initialize their secure memory.

[0018] The secure channel also has the ability to ensure the freshness of transmitted data by using an incrementing self-initialization vector for AES-GCM encryption.

[0019] Preferably, during initialization, the HSMEM transmission engine includes a meta-information selection module, specifically:

[0020] When data is transmitted through the HSMEM transmission channel, security metadata is transmitted together with the data. For the integrity tree BMT, the counter and the message authentication code MAC value are transmitted together. Each data transmission covers at least the entire subtree, so the minimum transmission granularity is the memory size covered by a single integrity tree node. The selected subtree satisfies that all counters in the subtree root node are related to the data to be transmitted. The counter in the parent node is called the subtree root counter for the subtree root node. Assume that the CPU and GPU use split counters in the integrity tree. The fan-out of the integrity tree node of the GPU is 128, and the cache line size is 128 bytes, while the degree of the CPU is 64 and the cache line size is 64 bytes. The granularity of CPU-GPU transmission is different from that of GPU-CPU transmission. The covered size is calculated by multiplying the degree by the cache line size. For CPU-GPU transmission, the granularity is 4KB, and only the data covered by the leaf nodes of the integrity tree will be transmitted;

[0021] The HSMEM transmission engine uses the address of the data to be transmitted to determine the integrity tree nodes that should be transmitted with the data, finds one or more subtrees covering the range of the transmitted data, and continues to search for subtrees at lower levels of the integrity tree if some counters in the root node do not belong to the data to be transmitted.

[0022] Preferably, the mode selection module includes: on the CPU or GPU, for each memory block, the memory controller knows the current mode of the block, and when the block with security metadata is transferred from the CPU to the GPU, it is still protected by the original protection scheme;

[0023] The mode selection module is a hardware module in the memory controller that is used to record such information. One bit is used to indicate the protection scheme of the memory block. The cache line size of the GPU is 128 bytes, while the cache line size of the CPU is 64 bytes. On the GPU, the size of the bitmap is 1 / 2^10 of the total memory size, and on the CPU, the size of the bitmap is 1 / 2^9 of the total memory size. For 1GB of GPU memory, the GPU needs 1MB of space for the bitmap, while for 1GB of system memory, the CPU needs 2MB.

[0024] Two levels of checking are used to reduce the overhead of accessing the bitmap. On the GPU, each bit of the first-level bitmap represents 128 bytes, while in the second-level bitmap, each bit represents 128KB. The first-level bitmap is stored directly in the on-chip buffer inside the memory controller. The second-level bitmap is stored in secure memory and integrity must be ensured. An area that is always in local mode is reserved to store the bitmap; on the GPU, this area is always in GPU mode, while on the CPU, it is always in CPU mode. A dedicated cache is provided for the second-level bitmap, called the bitmap cache, and the cache line size is also 128 bytes. Therefore, each bit stored in the cache line represents 128KB of memory. When a cache line is replaced from the bitmap cache, the value of the cache line is checked. If all bits in the replaced cache line are not set, which means that all 128KB are in GPU mode, the bits in the first-level bitmap will be unset. If any bit in the bitmap is set, the corresponding bits in both the first-level bitmap and the bitmap are set.

[0025] Preferably, the data encryption module includes: the CPU or GPU directly transmits data to each other without re-encryption, sets encryption or decryption memory blocks, encryption requires addresses and counters, if two blocks are encrypted using the same encryption key, address and counter, it is considered unsafe, therefore, each time a dirty memory block is evicted from the cache and written to the memory, the counter of the memory block increases, if the CPU and GPU use a shared encryption key, it is also necessary to maintain a unique address for each memory block on the CPU or GPU, therefore, the address used for memory encryption is different from the local physical address on the CPU or GPU;

[0026] Map all memories, including system memory and accelerator memory, to a single memory space called the encrypted address space EAS for encryption. The addresses in EAS are only used for encryption and decryption, and the addressing of normal memory access remains unchanged. EAS is implemented by providing a starting address for each device. During data transmission, the receiver needs additional information about the address used to decrypt the transmitted ciphertext;

[0027] The data encryption module first obtains the mode from the mode selection module, then obtains the correct counter from the integrity tree module, finds the correct EAS address, and then decrypts the data block;

[0028] The CPU uses half of the MAC to validate its own 64-byte cache line.

[0029] Preferably, the integrity tree module of the GPU mode includes: different memory controllers use their own integrity trees and verify their data without interacting with other memory partitions. In order to directly access the transmitted data, the GPU supports the protection schemes of the CPU mode and the GPU mode. The data of the CPU mode is encrypted using the address and counter from the CPU. When obtaining such a memory block, the memory controller obtains the correct counter value and address to decrypt the block. In order to support the CPU mode, a leaf node of the integrity tree involves blocks from different memory partitions. Therefore, a module is set to verify the counter and communicate the correct counter value to all memory controllers.

[0030] A CPU mode engine is used to verify the counters in the CPU mode integrity tree. The CPU mode engine is an on-chip module and is connected to other memory controllers. The transmitted CPU mode encryption-protected metadata is stored in the GPU memory. The CPU mode engine obtains the read memory by sending requests directly to the memory controller. The metadata will be directly obtained and returned to the CPU mode engine. When the memory controller needs to obtain the real counter value, it sends a request to the CPU mode engine. The CPU mode engine reads the integrity tree node stored in the GPU memory, verifies the counter, and returns the value to the memory controller.

[0031] Preferably, the integrity tree module of the CPU mode includes: the engine on the CPU side supports two different modes, in addition to the root node of the CPU mode counter, the memory controller on the CPU also reserves space to store the root node of the subtree in the GPU mode, the two different types of integrity trees are directly supported in a single memory controller of the CPU, and the EAS address of the subtree is also stored together with the count node in the cache, when the mode selection engine determines that a memory block is in the GPU mode, the multi-mode protection engine first verifies the counter in the GPU mode and the EAS address of the transmission block, and then decrypts and verifies the block, in order to support the GPU mode, the CPU maintains several integrity trees of the GPU mode;

[0032] All integrity trees have the same shape. During initialization, the GPU provides information describing the memory addressing scheme. Therefore, the CPU knows how the memory partition addresses are arranged and correctly uses the GPU's addressing scheme to find the integrity tree corresponding to the memory block.

[0033] Preferably, the data transmission process of the HSMEM transmission channel is:

[0034] Step 1: The software sends instructions to the GPU through the secure channel mechanism of GPU TEE;

[0035] Step 2: After reading the instruction, the GPU command processor sends a signal to the HSMEM transfer engine on the CPU side. This is implemented as a special interrupt request, which does not actually cause an external interrupt, but passes the signal and information to the HSMEM transfer engine on the CPU side.

[0036] Step 3: The HSMEM transfer engine on the CPU side receives the signal, reads the root node on-chip and verifies it, and sends a request to the memory controller to prohibit the system from writing to this address;

[0037] Step 4: After selecting the root node, the HSMEM transfer engine sends a request to the memory controller to prohibit writing to the area. If some tree nodes in the cache are dirty, they should be flushed back to the memory. Then it reads the root node of the subtree to be transferred and transfers the root node through the secure channel.

[0038] Step 5: The HSMEM transfer engine ensures that the GPU DMA engine is in secure mode and calculates the metadata address and performs secure memory transfer. These commands give the physical address of the range to be transferred, and this special memory transfer bypasses the IOMMU check. The address of the metadata area is given by the HSMEM transfer engine at system initialization. It directly calculates the address of the metadata and controls the DMA engine to extract the correct metadata. Data, MAC, and integrity tree nodes should be transferred. The above steps do not need to be re-encrypted.

[0039] Step 6: After reading the metadata from the system memory, the GPU sends another special signal to the CPU. This signal does not trigger a software interrupt, but is processed by the HSMEM transfer engine on the CPU side and sends the root node and EAS address of the subtree to the GPU through a secure channel.

[0040] Step 7: The HSMEM transmission engine on the CPU side sends a message to the memory controller to cancel the write restriction on the memory block. In the transmission channel, for CPU-GPU or GPU-CPU data transmission, HSMEM requires that all transmitted data be in the same mode. For GPU-GPU data transmission, MMT transmission mode is adopted, and data is protected by different modes.

[0041] Preferably, the memory access process of the multi-mode data protection engine is:

[0042] Step 11: Obtain the mode of the currently accessed block through the mode selection module. If it is the CPU mode, perform the following steps. If it is the GPU mode, use the normal GPU secure memory access mode.

[0043] Step 12: When such a block is fetched from DRAM, it is verified by the CPU mode integrity tree and verified and decrypted using the integrity tree module and memory encryption module;

[0044] Step 13: The decrypted and verified data is put into cache for access;

[0045] Step 14: After the access, if the block has not been modified, it is directly eliminated from the cache. If it has been modified, it is necessary to read the GPU mode counter, re-encrypt it, update the GPU mode metadata, and then write it back. When the GPU needs to encrypt the block and write it to the memory, it does not use the EAS address provided by the CPU, but uses its own EAS address to encrypt the block;

[0046] Step 15: If the mode is switched to GPU mode, the mode selection module is notified, and the mode selection module updates the corresponding information and marks the current memory block as changed back to GPU mode.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] (1) High-performance data transmission between heterogeneous secure memories (without re-encryption) is achieved, which reduces the time spent on CPU-GPU transmission and improves the overall performance of the application.

[0049] (2) Supporting multiple secure memory protection modes on a single secure memory device, so that previously transmitted data (still using the original secure memory protection mode) can be directly accessed by the device without the need to immediately switch modes;

[0050] (3) On a secure memory device, a memory block can efficiently switch between different secure memory protection modes. According to the needs of the application, it can be transparently converted to a higher-performance secure memory protection mode at runtime to achieve optimal performance on the current device. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0052] Figure 1 Intel SGX style memory integrity tree;

[0053] Figure 2 It is the memory encryption process;

[0054] Figure 3 Partition GPU memory;

[0055] Figure 4 For system architecture;

[0056] Figure 5 Metadata for selecting integrity tree;

[0057] Figure 6 It is a multi-mode protection engine. DETAILED DESCRIPTION

[0058] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0059] Example 1

[0060] System architecture or scenario of application of the present invention

[0061] HSMEM considers heterogeneous platforms for confidential computing. The protected application runs TEE on both the CPU side and the GPU side. Both the TEE on the CPU side and the GPU side use secure memory to protect data. HSMEM aims to build DRAM security for GPUTEE. For the management of TEE, HSMEM can use a mechanism similar to Gravition[OSDI'18]. Only the chip internal components of the CPU and GPU are trusted. Therefore, all data leaving the chip should be protected. The confidentiality, integrity and freshness of the data should be guaranteed. The present invention considers GPUs or other accelerators using GDDR memory, where an attacker can eavesdrop on the memory bus on the CPU or GPU, or the PCIe bus or other interconnect bus between the CPU and GPU. The attacker can also physically issue instructions to read, modify or replay the data in memory.

[0062] The core hardware modules of HSMEM include the HSMEM transfer engine and the multi-mode protection engine. The HSMEM transfer engine establishes an HSMEM transfer channel to achieve high-speed transmission between the CPU and the GPU. It is located inside the chip and controls special DMA requests for secure memory. It selects metadata related to the memory to be transferred and can transfer data without additional data encryption and decryption. For each data transfer, only a few cache blocks need to be encrypted, so it can provide a speed close to that of plaintext transmission. After the data is transferred to the CPU or GPU, the multi-mode protection engine is located in the memory controller and can directly use the transmission data of the original protection scheme. The engine can support multiple protection schemes. In the present invention, each memory protection scheme supported by the multi-mode protection engine is referred to as a "mode". A memory block can be protected by different modes. Inside the multi-mode protection engine, there is a mode selection module for selecting a mode for a memory block. There is also a data encryption module for data encryption and decryption, and an integrity tree module for maintaining integrity trees of different modes. In addition, HSMEM is not limited to protection schemes similar to BMT, it can be extended to other memory protection schemes with looser security requirements. HSMEM also provides an interface that allows developers to explicitly change the mode of a memory block using dedicated instructions.

[0063] The core network element / device module / product of the present invention is implemented, and the overall system architecture is as follows Figure 4 shown.

[0064] The HSMEM transfer engine builds an HSMEM transfer channel for secure direct memory access (DMA). The HSMEM transfer channel enables high-speed data transfer between different devices equipped with secure memory. During data transfer, there is no need to re-encrypt, only data and security metadata need to be transferred. During initialization, the HSMEM transfer engine generates keys with the peer device and builds a traditional secure channel. This secure channel is part of the HSMEM transfer channel. When data is to be transferred, the HSMEM transfer engine prepares the metadata that needs to be transferred with the data. In addition, the secure channel is used to exchange some basic system configurations during the initialization process. The HSMEM transfer engine calculates the target address of the metadata corresponding to the data, and then the HSMEM transfer engine controls the DMA engine to copy the metadata to the target location. The GPU uses its own DMA engine for data transfer. The transfer engine contains: an initialization module and a metadata selection module.

[0065] Initialization module: When the system boots up or a GPU is plugged in, the CPU and GPU should authenticate each other and establish a traditional secure channel. Through the secure channel, the CPU and GPU can negotiate multiple shared keys for memory encryption. In addition, the CPU and GPU provide each other with memory layout information. They should provide the integrity tree format, the type of encryption algorithm, and the address of the metadata area. The GPU should also provide some additional information to the CPU, including the number of memory controllers and the memory addressing scheme. After this is completed, the CPU and GPU will initialize their secure memory. The secure channel should also have the ability to ensure the freshness of the transmitted data. This can be achieved by using an incrementing self-initialization vector (IV) for AES-GCM encryption. It should be noted that maintaining such a secure channel does not impose a large burden on the HSMEM transmission engine, and even the PCIe security enhancements that already use AES-GCM to maintain such a secure channel can be reused.

[0066] Meta-information selection module: When data is transmitted through the HSMEM transmission channel, some security metadata is transmitted with the data. For BMT, counters and MAC values ​​are transmitted together. HSMEM requires that each data transmission covers at least the entire subtree, so the minimum transmission granularity is the memory size covered by a single integrity tree node. The selected subtree must satisfy that all counters in the subtree root node are related to the data to be transmitted. In the present invention, the counter in the parent node is called the subtree root counter for the subtree root node. Here, it is assumed that the CPU and GPU use split counters in the integrity tree. The integrity tree node fan-out of the GPU is 128 and the cache line size is 128 bytes, while the degree of the CPU is 64 and the cache line size is 64 bytes. The granularity of CPU-GPU transmission is different from that of GPU-CPU transmission. The size covered can be calculated by multiplying the degree by the cache line size. For CPU-GPU transmission, the granularity is 4KB, which means that only the data covered by the leaf nodes of the integrity tree will be transmitted. First, the HSMEM transmission engine uses the address of the data to be transmitted to determine the integrity tree node that should be transmitted with the data. It finds one or more subtrees that cover the range of the data to be transmitted. It should ensure that all the data covered in the root node of the selected subtree is the data to be transmitted. If some counters in the root node do not belong to the data to be transmitted, it should continue to search for subtrees at lower levels of the integrity tree, such as Figure 5 shown.

[0067] GPU-CPU data transfer uses different granularity. Due to the address arrangement of GPU memory, the cache lines of adjacent memory controllers may not be adjacent. However, to simplify the metadata management of data transfer on the CPU, HSMEM ensures that at least one memory controller's integrity tree is transferred and all memory controllers transfer the same amount of data to the CPU. Therefore, the GPU mode integrity tree maintained on the CPU is isomorphic and at least NR_MemCtrl root nodes need to be transferred. The granularity of the transfer also depends on the address arrangement granularity. NR_MemInter is the memory address arrangement granularity, indicating that adjacent NR_MemInter bytes come from the same memory partition. The granularity of the copy is min(NR_MemInter, 16KB)*NR_MemCtrl, that is, all memory controllers transfer the same amount of data to the CPU. Assuming the memory address arrangement granularity is 2 memory blocks (256 bytes) and there are 16 memory controllers, the granularity of the copy should be 256KB.

[0068] The multi-mode protection engine supports multiple memory protection schemes in a device equipped with secure memory. The multi-mode protection engine of both the CPU and GPU supports two memory protection schemes. When the CPU or GPU receives data through the HSMEM transmission channel, the data is first protected by its original protection scheme. The original mode is used when reading data, and the local mode is switched to when writing data. First, this solution describes how to read CPU mode memory on the GPU, and how to read GPU mode memory on the CPU, and then describes how to switch between the two memory modes. The architecture of the multi-mode protection engine is as follows: Figure 6 The CPU-mode integrity tree and MAC on the GPU are called shadow metadata. Similarly, the GPU-mode security metadata on the CPU is also called shadow metadata.

[0069] Mode selection module: On the CPU or GPU, for each memory block, the memory controller should know the current mode of the block. When a block with security metadata is transferred from the CPU to the GPU, it is still protected by the original protection scheme, and vice versa. The mode selection module is a hardware module in the memory controller that records such information. It uses one bit to indicate the protection scheme (i.e., mode) of the memory block. The cache line size of the GPU is 128 bytes, while the cache line size of the CPU is 64 bytes. On the GPU, the size of the bitmap is 1 / 2^10 of the total memory size, and on the CPU, the bitmap size is 1 / 2^9 of the total memory size. For 1GB of GPU memory, the GPU requires 1MB of space for the bitmap, while for 1GB of system memory, the CPU requires 2MB. However, the current memory of existing CPUs or GPUs ranges from tens to hundreds of GB. It is not practical to store all bitmaps on the chip. However, each memory access should be checked by the bitmap. HSMEM uses two-level checks to reduce the overhead of accessing the bitmap. On the GPU, each bit of the first-level bitmap represents 128 bytes, while in the second-level bitmap, each bit represents 128KB. The first-level bitmap is stored directly in an on-chip buffer inside the memory controller. The second-level bitmap is stored in secure memory and its integrity must be ensured. HSMEM reserves an area that is always in local mode to store the bitmap.

[0070] On the GPU, this region must always be in GPU mode, while on the CPU, it must always be in CPU mode. A dedicated cache is provided for the second-level bitmap, called the bitmap cache, and the cache line size is also 128 bytes (1028 bits). Therefore, each bit stored in the cache line represents 128KB of memory. When a cache line is replaced from the bitmap cache, the value of the cache line is checked. If all bits in the replaced cache line are not set, which means that all 128KB are in GPU mode, then the bits in the first-level bitmap will be unset. If any bit in the bitmap is set, the corresponding bits in both the first-level bitmap and the bitmap should be set. Each memory controller on the GPU has a mode selection module to manage its memory partitions. The mode selection module on the CPU is similar, but each bit in the first-level bitmap represents 64 bytes, while each bit in the first-level bitmap represents 64KB.

[0071] Data Encryption Module: HSMEM allows the CPU or GPU to transfer data directly to each other without re-encryption. To encrypt or decrypt a memory block, encryption requires an address and a counter, such as Figure 2As shown. If two blocks are encrypted using the same encryption key, address, and counter, it is considered insecure because the same One-time Pad (OTP) is used for both blocks. Therefore, every time a dirty memory block is evicted from the cache and written to the memory, the counter of the memory block is increased. Similar to MMT (see Related Art), if the CPU and GPU use a shared encryption key, HSMEM also needs to maintain a unique address for each memory block on the CPU or GPU. Therefore, the address used for memory encryption is different from the local physical address on the CPU or GPU. HSMEM maps all memory in the system, including system memory and accelerator memory, to a single memory space called the Encrypted Address Space (EAS) for encryption. The addresses in the EAS are only used for encryption and decryption, and the addressing of normal memory access remains unchanged. EAS can be easily implemented by providing a starting address for each device. At the time of data transmission, the receiver needs additional information about the address used to decrypt the transmitted ciphertext. The data encryption module should first obtain the mode from the mode selection module, then obtain the correct counter from the integrity tree module, and find out the correct EAS address. The data block can then be decrypted. Since the GPU cache line size is 128 bytes, which is twice the size of the CPU cache line. In the GPU mode integrity tree on the CPU, two adjacent CPU memory blocks use the same counter. On the CPU, if a memory block is obtained, the counter of the GPU integrity tree is verified and used for decryption on the CPU. The EAS address is also stored with the node of the integrity tree. Although the two CPU cache lines use the same GPU MAC, the CPU can only read one of the cache lines. This is because HSMEM changes the way the GPU calculates MAC, and the GPU MAC is generated by concatenating two MACs. Each MAC is generated from one of the 64 bytes in the GPU cache line. Therefore, the CPU can use half of the MAC to verify its own 64-byte cache line.

[0072] Integrity Tree Module (GPU Mode): GPUs with confidential computing capabilities naturally support GPU mode protection schemes. For GPU mode integrity trees, different memory controllers can use their own integrity trees and can verify their data without interacting with other memory partitions. In order to directly access the transmitted data, the GPU should support CPU mode and GPU mode protection schemes. CPU mode data is encrypted using addresses and counters from the CPU. When such a memory block is fetched, the memory controller should fetch the correct counter value and address to decrypt the block. However, to support CPU mode, one leaf node of the integrity tree involves blocks from different memory partitions. Therefore, there should be a module to verify the counter and communicate the correct counter value to all memory controllers. HSMEM uses a dedicated unit, the CPU mode protection engine, to verify the counters in the CPU mode integrity tree. The CPU mode engine is an on-chip module and interfaces with other memory controllers. The transmitted CPU mode encryption protection metadata is stored in the GPU memory. The CPU mode engine can fetch the read memory by sending a request directly to the memory controller. The metadata will be directly fetched and returned to the CPU mode engine. When the memory controller needs to get the real counter value, it sends a request to the CPU mode engine. The CPU mode engine reads the integrity tree node stored in GPU memory and verifies the counter and returns the value to the memory controller.

[0073] As introduced earlier, several subtrees are transferred each time data is transferred to the GPU. The root counter of the subtree is protected by the secure channel. After a subtree is transferred, the root counter is stored in the on-chip part of the accelerator. The multi-mode protection engine logically uses the CPU mode integrity tree to protect all secure memories. The physical address of the memory block is used to determine which CPU mode counter the block is associated with. When there is no data transfer, there is no valid node in the CPU mode integrity tree. When data is transferred from the CPU, the corresponding node in the CPU mode integrity tree will become a valid node according to the physical address of the transferred data. The transferred subtree will be placed directly in the corresponding position. However, since the subtree has no valid parent node, the root counter of the subtree will be stored on the chip. Each CPU mode root counter requires up to 8 bytes. The multi-mode protection engine only needs to store the counter value in the parent counter. At the same time, the block address on the CPU or the block address in the EAS is also transferred and stored on the chip together with the root counter of the subtree. Therefore, for each subtree, only one counter and the 64-bit EAS address need to be stored. When the counter is needed, all tree nodes on the path to the root counter are verified. This continues until the first node in the on-chip cache is reached. In the CPU mode counter cache on the accelerator, each node is stored with the ID of the root node stored on the chip. When verifying a node of the Counter Cache, its corresponding EAS address is obtained based on the EAS address of the root node. The accelerator is not a regular processor core and only has fixed logic about the CPU mode protection scheme, so more on-chip storage can be used. When the GPU memory controller fetches a memory block, it can request the EAS address and counter value from the accelerator. The memory controller can then verify and decrypt the fetched memory block. However, if there are too many subtrees, the on-chip buffer cannot store the EAS addresses and counters of these subtrees. HSMEM provides two different methods. The first method is that the unit sends an interrupt to the GPU command processor. The GPU command processor will then start a special GPU to convert a range of memory to local GPU mode. The second method is that these nodes can be evicted and saved in a region of secure memory. If the node is not found in the cache, the engine will fetch the node from protected memory.

[0074] Integrity Tree Module (CPU Mode): The CPU-side engine should also support two different modes. In addition to the root node of the CPU mode counter, the memory controller on the CPU should also leave space to store the root node of the subtree in GPU mode. These two different types of integrity trees can be supported directly in a single memory controller of the CPU. The EAS address of the subtree should also be stored with the counter node in the cache. When the mode selection engine determines that a memory block is in GPU mode, the multi-mode protection engine first verifies the counter in GPU mode and the EAS address of the transfer block. The block can then be decrypted and verified. To support GPU mode, the CPU maintains several integrity trees for GPU mode. Since the memory address arrangement is fixed, the CPU does not need to store EAS addresses for all integrity trees in GPU mode. All integrity trees have the same shape because the transfer granularity selected by this scheme ensures that the integrity trees of all GPU modes have the same shape. At initialization, the GPU provides information that describes the memory addressing method. Therefore, the CPU knows the memory partition address arrangement method and can correctly use the GPU's addressing scheme to find the integrity tree corresponding to the memory block.

[0075] Data transfer process of HSMEM transfer channel: When interacting with the CPU, in addition to reading or writing MMIO registers, the GPU kernel space driver (module in the operating system kernel) or user space driver (CUDA driver) usually sends commands through the GPU channel. Using a GPU-TEE mechanism like Graviton, the integrity and confidentiality of the commands have been protected. When the driver pushes a command through the GPU channel to request data transfer with the HSMEM transfer engine, the address range of the memory range to be transferred should also be provided. The address translation hierarchy of the GPU needs to be considered. The GPU has its own page table, and the address used by the GPU kernel is the GPU virtual address (VA). The GPUMMU converts the address to the GPU physical address (PA). The page in the GPU memory or system is marked in the GPU page table. In current GPUs, there are two types of GPU channels with different permissions: privileged channels and non-privileged channels. The kernel space driver can use the privileged channel, and the commands in the privileged channel can directly use the physical address. For user space drivers, the non-privileged channel is used. The pushed commands can only use GPUVA, but will eventually be converted to GPUPA. However, in modern computer systems, IOMMU is usually used to protect the device from arbitrary access to system memory. The GPU cannot directly observe the physical address in IOPA or system address. We note that in modern GPUs, the Address Translation Service (ATS) is used. For some supercomputers that support CPU-GPU NVLINK interconnect (such as Summit Super Computer), ATS enables the GPU to access system memory using CPU virtual addresses. When ATS is enabled, the device or GPU will send a request to the address translation agent on the CPU side to request the address corresponding to the IOPA of a certain address. The agent will translate the address and return the IOPA to the GPU. The GPU uses the Address Translation Service (ATC) to cache address translations if an entry is found in the ATC. IOPA can be used directly, bypassing the IOMMU to reduce the pressure on the IOTLB in the IOMMU. HSMEM transfers can directly use ATS to determine the real address of the transferred data. Secure transfer requests require the transfer range to be continuous, so only the address of the first byte is required.

[0076] Step 1: Through GPUTEE’s secure channel mechanism, the software sends commands to the GPU.

[0077] Step 2: After reading the instruction, the GPU command processor sends a signal to the HSMEM transfer engine on the CPU side. This can be implemented as a special interrupt request, which does not actually cause an external interrupt, but passes the signal and information to the HSMEM transfer engine on the CPU side;

[0078] Step 3: The HSMEM transfer engine on the CPU side receives the signal, reads the root node on-chip and verifies it, and sends a request to the memory controller to prohibit the system from writing to this address;

[0079] Step 4: After selecting the root node, the HSMEM transfer engine should send a request to the memory controller to prohibit writing to the area. If some tree nodes in the cache are dirty, they should be flushed back to the memory. Then it reads the root node of the subtree to be transferred and transfers the root node through the secure channel.

[0080] Step 5: The HSMEM transfer engine ensures that the GPU DMA engine is in secure mode and calculates the metadata address and performs secure memory transfer. These commands give the physical address of the range to be transferred, and this special memory transfer can bypass the IOMMU check. The address of the metadata area is given by the HSMEM transfer engine when the system is initialized. It can directly calculate the address of the metadata and control the DMA engine to extract the correct metadata. Data, MAC, and integrity tree nodes should be transferred. The above steps do not need to be re-encrypted.

[0081] Step 6: After reading the metadata from the system memory, the GPU sends another special signal to the CPU. This signal does not trigger a software interrupt, but is processed by the HSMEM transfer engine on the CPU side and sends the root node and EAS address of the subtree to the GPU through a secure channel.

[0082] Step 7: The HSMEM transfer engine on the CPU side sends a message to the memory controller to cancel the write restriction on the block of memory. In the transmission channel, for CPU-GPU or GPU-CPU data transmission, HSMEM requires that all transmitted data be in the same mode. For example, if the GPU needs to transfer data to the CPU, then all data should be protected by GPU mode or CPU mode. For GPU-GPU data transmission, using the MMT transmission method, data can be protected by different modes, which means that some data can be protected by CPU mode and some data can be protected by GPU mode.

[0083] Memory access process of the multi-mode data protection engine: Taking the GPU side as an example, when a memory block is in CPU mode, the GPU integrity tree remains unchanged. The counter node is still retained in the memory in plain text, the same as the original integrity tree structure. The CPU side implementation is similar.

[0084] Step 1: Obtain the mode of the currently accessed block through the mode selection module. If it is CPU mode, perform the following steps. If it is GPU mode, use the normal GPU secure memory access mode.

[0085] Step 2: When such a block is fetched from DRAM, it is verified by the CPU mode integrity tree and validated and decrypted using the integrity tree module and memory encryption module;

[0086] Step 3: The decrypted and verified data is put into cache for access;

[0087] Step 4: After the access, if the block has not been modified, it can be directly eliminated from the cache. If it has been modified, it is necessary to read the GPU mode counter, re-encrypt it, update the GPU mode integrity tree and other metadata, and then write it back. When the GPU needs to encrypt the block and write it to the memory, it will not use the EAS address provided by the CPU, but use its own EAS address to encrypt the block;

[0088] Step 5: If the mode is switched to GPU mode, the mode selection module is notified, and the mode selection module updates the corresponding information and marks the current memory block as changed back to GPU mode.

[0089] Example 2

[0090] Embodiment 2 is a preferred example of Embodiment 1.

[0091] The present invention can be implemented on a GPU and a CPU using GDDR, and can support a memory protection mode with an integrity tree, which can defend against physical replay attacks. At the same time, the CPU and GPU themselves have a TEE mechanism, and this solution can be built on the original TEE mechanism to achieve efficient data transmission. At the same time, the present invention provides special instructions, and users can write special GPUKernel, using special instructions to actively perform memory conversion without putting data into the GPU cache. The instruction will directly send a request to the memory controller, and the memory controller will directly convert the memory mode without putting the converted result into the cache.

[0092] At the same time, caches dedicated to storing integrity tree nodes and data MACs are added to different memory controllers to accelerate access to memory protection metadata. In the implementation process, multiple integrity trees can be used to protect memory in the same memory controller, reducing the depth of the tree, similar to MMT.

[0093] This embodiment can achieve high-performance data transmission between the CPU and the GPU and ensure data security while protecting the confidentiality, integrity, and freshness of the secure memory, and when the CPU secure memory and the GPU secure memory are heterogeneous.

[0094] Example 3

[0095] Embodiment 3 is a preferred example of Embodiment 1.

[0096] HSMEM can support different protection schemes. In Example 1 and Example 2, the strongest protection scheme is mainly used to ensure confidentiality, integrity and freshness at the same time to resist physical attacks. This can be achieved by a design based on Bonsai MerkleTree (BMT), such as Intel SGX. However, some hardware platforms may not be able to provide security guarantees. For example, under physical attacks, AMD SEV does not guarantee integrity and freshness, while Intel TDX does not consider physical replay attacks. Therefore, the integrity tree and MAC can be deleted in such architectures.

[0097] By customizing the multi-mode protection engine, HSMEM can be extended to other protection schemes. In this architecture, the integrity tree is not required. Because the address is still used as an adjustment value for encryption, the receiver still needs to transmit and record the address when the data is transmitted. For the transmission block that is not encrypted with the local EAS address, the multi-mode protection engine should have some mechanism to find the correct EAS address to decrypt the block. HSMEM uses the EAS translation engine to obtain the decrypted EAS address. For large memory transfers, only recording the starting address of the original EAS address is sufficient for decryption. Therefore, the EAS translation engine maintains some range registers, and when the data is transmitted, it records the starting address, ending address, and offset of the data. This information is stored in the range buffer. When the mode selection module finds that the memory block does not belong to the current mode, the memory controller checks whether the address falls into any range in the range buffer. However, the size of the range buffer is limited. The EAS translation engine also maintains a multi-level translation table for the conversion of EAS addresses. The table converts the local address to the original EAS address with a granularity of 4K. The table is organized similarly to a page table and can also support a mechanism similar to large pages. The EAS translation engine scans the bitmap periodically and removes entries in the range buffer if all memory is switched to local mode. If the range buffer does not have enough space to store the range, it evicts an entry and updates the mapping table. The engine checks if the new range is larger than the existing range in the buffer, and the old range is evicted only if the new range is larger than the existing range. Since data is usually transferred in large blocks, large pages can still be used even if the range buffer is full and the translation table is used, thus reducing overhead. The mapping table can introduce a TLB-like buffer and can be combined with the range buffer, and the overhead introduced by the translation is limited.

[0098] The present invention can also be used in other accelerators that do not use HBM memory.

[0099] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0100] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A high-performance heterogeneous secure memory, characterized in that: include: HSMEM transport engine, multi-mode protection engine and interface; The HSMEM transmission engine establishes an HSMEM transmission channel to achieve high-speed transmission between the CPU and the GPU. The HSMEM transmission engine is located inside the CPU and GPU chips and controls DMA requests for secure memory. The HSMEM transmission engine selects metadata related to the memory to be transmitted and transmits data without additional data encryption and decryption. For each data transmission, a preset number of cache blocks are encrypted. After the data is transmitted to the CPU or GPU, the multi-mode protection engine is located in the memory controller and directly uses the original protection scheme to transmit data; Each memory protection scheme supported by the multi-mode protection engine is called a mode. A memory block is protected by different modes. Inside the multi-mode protection engine, there is a mode selection module for selecting a mode for a memory block, a data encryption module for data encryption and decryption, and an integrity tree module for maintaining integrity trees of different modes. The integrity tree module of GPU mode includes: different memory controllers use their own integrity trees and verify their data without interacting with other memory partitions. In order to directly access the transmitted data, GPU supports the protection schemes of CPU mode and GPU mode. The data of CPU mode is encrypted using the address and counter from the CPU. When obtaining such a memory block, the memory controller obtains the correct counter value and address to decrypt the block. In order to support CPU mode, a leaf node of the integrity tree involves blocks from different memory partitions. Therefore, a module is set to verify the counter and communicate the correct counter value to all memory controllers. The HSMEM transmission engine uses a CPU mode protection engine in the multi-mode protection engine to verify the counter in the CPU mode integrity tree. The CPU mode protection engine is an on-chip module and is connected to other memory controllers. The transmitted CPU mode encryption protection metadata is stored in the GPU memory. The CPU mode protection engine obtains the read memory by directly sending a request to the memory controller. The metadata will be directly obtained and returned to the CPU mode protection engine. When the memory controller needs to obtain the real counter value, it sends a request to the CPU mode protection engine. The CPU mode protection engine reads the integrity tree node stored in the GPU memory, verifies the counter, and returns the value to the memory controller. The integrity tree module of CPU mode includes: the multi-mode protection on the CPU side supports two different modes. In addition to the root node of the CPU mode counter, the memory controller on the CPU also reserves space to store the root node of the subtree in GPU mode. These two different types of integrity trees are directly supported in a single memory controller of the CPU. The EAS address of the subtree is also stored together with the count node in the cache. When the mode selection engine determines that a memory block is in GPU mode, the multi-mode protection engine first verifies the counter in GPU mode and the EAS address of the transfer block, and then decrypts and verifies the block. In order to support GPU mode, the CPU maintains several integrity trees for GPU mode; All integrity trees have the same shape. During initialization, the GPU provides information about the memory addressing scheme. Therefore, the CPU knows how the memory partition addresses are arranged and correctly uses the GPU's addressing scheme to find the integrity tree corresponding to the memory block. Through the interface, developers use preset instructions to change the mode of a memory block.

2. The high-performance heterogeneous secure memory according to claim 1, characterized in that: During initialization, the HSMEM transmission engine generates keys with the peer device and builds a traditional secure channel. When data is to be transmitted, the HSMEM transmission engine prepares metadata that needs to be transmitted along with the data. In addition, the secure channel is used to exchange some basic system configurations during the initialization process. The HSMEM transfer engine calculates the target address of the metadata corresponding to the data, and then the HSMEM transfer engine controls the DMA engine to copy the metadata to the target location. The GPU uses its own DMA engine for data transfer.

3. The high-performance heterogeneous secure memory according to claim 1, characterized in that: During initialization, the HSMEM transmission engine includes an initialization module, specifically: When the system boots up or the GPU is plugged in, the CPU and GPU authenticate each other and establish a traditional secure channel. Through the secure channel, the CPU and GPU negotiate multiple shared keys for memory encryption. In addition, the CPU and GPU provide each other with memory layout information, including the integrity tree format, encryption algorithm type, and the address of the metadata area. The GPU also provides the CPU with the number of memory controllers and the memory addressing scheme. After completing this operation, the CPU and GPU will initialize their secure memory. The secure channel also has the ability to ensure the freshness of transmitted data by using an incrementing self-initialization vector for AES-GCM encryption.

4. The high-performance heterogeneous secure memory according to claim 1, characterized in that: During initialization, the HSMEM transmission engine includes a meta-information selection module, specifically: When data is transmitted through the HSMEM transmission channel, security metadata is transmitted together with the data. For the integrity tree BMT, the counter and the message authentication code MAC value are transmitted together. Each data transmission covers at least the entire subtree, so the minimum transmission granularity is the memory size covered by a single integrity tree node. The selected subtree satisfies that all counters in the subtree root node are related to the data to be transmitted. The counter in the parent node is called the subtree root counter for the subtree root node. The CPU and GPU use split counters in the integrity tree. The fan-out of the integrity tree node of the GPU is 128, and the cache line size is 128 bytes, while the degree of the CPU is 64 and the cache line size is 64 bytes. The granularity of CPU-GPU transmission is different from that of GPU-CPU transmission. The size covered is calculated by multiplying the degree by the cache line size. For CPU-GPU transmission, the granularity is 4KB, and only the data covered by the leaf nodes of the integrity tree will be transmitted; The HSMEM transmission engine uses the address of the data to be transmitted to determine the integrity tree nodes that should be transmitted with the data, finds one or more subtrees covering the range of the transmitted data, and continues to search for subtrees at lower levels of the integrity tree if some counters in the root node do not belong to the data to be transmitted.

5. The high-performance heterogeneous secure memory according to claim 1, characterized in that: The mode selection module includes: on the CPU or GPU, for each memory block, the memory controller knows the current mode of the block, and when the block with security metadata is transferred from the CPU to the GPU, it is still protected by the original protection scheme; The mode selection module is a hardware module in the memory controller. It uses one bit to indicate the protection scheme of the memory block. The cache line size of the GPU is 128 bytes, while the cache line size of the CPU is 64 bytes. On the GPU, the size of the bitmap is 1 / 2^10 of the total memory size, and on the CPU, the size of the bitmap is 1 / 2^9 of the total memory size. For 1GB of GPU memory, the GPU needs 1MB of space for the bitmap, while for 1GB of system memory, the CPU needs 2MB. Two levels of checking are used to reduce the overhead of accessing the bitmap. On the GPU, each bit of the first-level bitmap represents 128 bytes, while in the second-level bitmap, each bit represents 128KB. The first-level bitmap is stored directly in the on-chip buffer inside the memory controller. The second-level bitmap is stored in secure memory and its integrity must be ensured. An area that is always in local mode is reserved to store the bitmap. On the GPU, this area is always in GPU mode, while on the CPU, it is always in CPU mode. A dedicated cache is provided for the second-level bitmap, called the bitmap cache, and the cache line size is also 128 bytes. Therefore, each bit stored in the cache line represents 128KB of memory. When a cache line is replaced from the bitmap cache, the value of the cache line is checked.

6. The high-performance heterogeneous secure memory according to claim 1, characterized in that: The data encryption module includes: CPU or GPU directly transmits data to each other without re-encryption, sets encryption or decryption memory blocks, encryption requires addresses and counters, if two blocks are encrypted using the same encryption key, address and counter, it is considered unsafe, therefore, each time a dirty memory block is evicted from the cache and written to the memory, the counter of the memory block increases, if the CPU and GPU use a shared encryption key, it is also necessary to maintain a unique address for each memory block on the CPU or GPU, therefore, the address used for memory encryption is different from the local physical address on the CPU or GPU; Map all memories, including system memory and accelerator memory, to a single memory space called the encrypted address space EAS for encryption. The addresses in EAS are only used for encryption and decryption, and the addressing of normal memory access remains unchanged. EAS is implemented by providing a starting address for each device. During data transmission, the receiver needs additional information about the address used to decrypt the transmitted ciphertext; The data encryption module first obtains the mode from the mode selection module, then obtains the correct counter from the integrity tree module, finds the correct EAS address, and then decrypts the data block; The CPU uses half of the MAC to validate its own 64-byte cache line.

7. The high-performance heterogeneous secure memory according to claim 1, characterized in that: The memory access process of the multi-mode protection engine is: Step 11: Obtain the mode of the currently accessed block through the mode selection module. If it is the CPU mode, perform the following steps. If it is the GPU mode, use the normal GPU secure memory access mode. Step 12: When the currently accessed block is fetched from DRAM, it is verified by the CPU mode integrity tree and verified and decrypted using the integrity tree module and memory encryption module; Step 13: The decrypted and verified data is put into cache for access; Step 14: After the access, if the block has not been modified, it is directly eliminated from the cache. If it has been modified, it is necessary to read the GPU mode counter, re-encrypt it, update the GPU mode metadata, and then write it back. When the GPU needs to encrypt the block and write it to the memory, it does not use the EAS address provided by the CPU, but uses its own EAS address to encrypt the block; Step 15: If the mode is switched to GPU mode, the mode selection module is notified, and the mode selection module updates the corresponding information and marks the current memory block as changed back to GPU mode.

Images