Dynamically allocable, physically addressable metadata storage

Abstract

In an example, there is a computing device comprising a processor, the processor having a memory management unit. The computing device also has a memory storing instructions that, when executed by the processor, cause the memory management unit to receive a memory access instruction including a virtual memory address, convert the virtual memory address to a physical memory address of the memory, and obtain permission information associated with the physical memory address. In response to the permission information indicating that the metadata is permitted to be associated with the physical memory address, a metadata summary table stored in the physical memory is checked to see if the metadata is compatible with the physical memory address. In response to the check being negative, a trap is sent to system software of the computing device to trigger dynamic allocation of physical memory for storing the metadata associated with the physical memory address.

Description

Background Art

[0001] Some hardware security schemes involve associating metadata with physical memory and checking the associated metadata to apply certain security policies or change the operation of the system when data is read from or written to the physical memory.

[0002] The embodiments described below are not limited to embodiments that solve some or all of the drawbacks of known techniques that store and use metadata in physical memory.

Summary of the Invention

Problems to be Solved by the Invention

[0003] A simplified summary of the present disclosure is presented below to provide the reader with a basic understanding. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Its sole purpose is to present, in a simplified form, a selection of concepts disclosed herein as an introduction to the more detailed description that follows.

Means for Solving the Problems

[0004] In various examples, there is a computing device equipped with a processor, which has a memory management unit. The computing device also has memory that stores instructions that, when executed by the processor, cause the memory management unit to receive memory access instructions containing virtual memory addresses, translate the virtual memory addresses to physical memory addresses, and retrieve information associated with the physical memory addresses, which is one or more of permission information and memory type information. Depending on whether this information indicates that metadata is permitted to be associated with a physical memory address, a metadata summary table stored in physical memory is checked to determine whether the metadata is compatible with the physical memory address.

[0005] Depending on whether the confirmation is negative, a trap is sent to the computing device's system software to allow the dynamic allocation of physical memory to store metadata associated with the physical memory address. In some examples, the metadata summary table is cacheable in the processor's cache.

[0006] Many of the associated features will be more easily understood by referring to the following detailed description, which is taken into consideration in conjunction with the attached drawings.

[0007] This explanation will be better understood from the following detailed explanation, which should be read in reference to the attached drawings. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a schematic diagram of a data center, showing the physical memory of the data center's compute nodes, which store data and metadata separately. [Figure 2] Figure 2 is a schematic diagram of the central processing unit, cache lines, virtual memory, and physical memory of a computing device, where physical memory stores both data and metadata. [Figure 3] Figure 3 is a schematic diagram of the central processing unit, cache lines, virtual memory, and physical memory of a computing device, where physical memory stores data and metadata separately. [Figure 4] Figure 4 is a schematic diagram of the memory management unit and tag controller of the computing device. [Figure 5] Figure 5 is a flowchart illustrating the process performed by the memory management unit. [Figure 6] Figure 6 is a flowchart illustrating how the system software performs the actions. [Figure 7] Figure 7 is a flowchart illustrating the process performed by the tag controller for cache line eviction. [Figure 8] Figure 8 is a flowchart illustrating the process performed by the tag controller for cache filling. [Figure 9] Figure 9 illustrates an exemplary computing-based device in which the embodiment is implemented. [Modes for carrying out the invention]

[0009] In the attached drawings, the same reference numerals are used to indicate similar parts.

[0010] The detailed description provided below in relation to the attached drawings is intended to describe this example and is not intended to represent the only form in which this example may be constructed or used. This description outlines the function of the example and the sequence of actions for constructing and deploying it. However, the same or equivalent function and sequence may be achieved by different examples.

[0011] In this book, the term "page" is used to refer to a unit of memory that is the same as or different from the unit of memory used by the memory management unit.

[0012] The examples described herein describe a processor that is a CPU. These examples also work for any processor that is connected to a cache-coherent interconnect such as an accelerator.

[0013] As mentioned earlier, some hardware security schemes involve associating metadata with physical memory, allowing the associated metadata to be reviewed to apply specific security policies when data is read from or written to physical memory. Examples of such hardware security schemes include, but are not limited to, Memory Tag Extensions (MTEs) and Functional Hardware Extensions RISC Instructions (CHERIs). Metadata associated with physical memory can also be used for purposes other than hardware security schemes.

[0014] Common hardware security schemes, which involve associating metadata with physical memory, require partitioning a certain range of physical memory to store the metadata. The inventors recognized that in cloud deployments, where compute nodes partition a range of physical memory to store metadata associated with physical memory for hardware security schemes or other purposes, an unacceptable memory overhead arises. In the case of MTEs, even if only a single virtual machine (VM) on a server is utilizing the metadata, the memory overhead can amount to 1 / 32 of the total memory.

[0015] In various embodiments, there is a mechanism that includes a fixed allocation of physical memory (much smaller, typically 1 / 512) for storing a metadata summary table, which can store coarse-grained metadata if needed, and a pointer to dynamically allocated physical memory for storing fine-grained metadata. Implementing such a metadata summary table is not straightforward because it must keep processing overhead low and have as little impact as possible on operations that do not use metadata. In embodiments, there is a mechanism for deploying a metadata summary table for storing fine-grained metadata, dynamic allocation of physical memory, and a set of software abstractions to manage it, in a typical central processing unit (CPU) design.

[0016] Figure 1 is a schematic diagram of data center 100, showing the physical memory 114 of the compute node 104 of data center 100, where the physical memory 114 stores data and metadata associated with that data separately. Data center 100 comprises multiple compute nodes 102, 104. Each compute node has a central processing unit (CPU) and physical memory 114. Compute nodes 102, 104 optionally communicate with each other via a communication network. The compute nodes are used to provide cloud computing services by running applications and providing application functionality to client devices via a communication network such as the Internet.

[0017] In this example, there are a first set of compute nodes 102 and a second set of compute nodes 102. Each compute node 104 has only virtual machines that use metadata. In contrast, each compute node 102 has at least one virtual machine that does not use metadata.

[0018] Figure 1 shows enlarged views of the physical memory of one of the computing nodes, using the first approach (left) and the second approach (right). In both the first and second approaches, data and the metadata associated with that data are stored separately in physical memory. "Stored separately" means that the memory locations of a given data and its associated metadata are not necessarily adjacent.

[0019] According to the first approach (left), a range of physical memory 106 is reserved within physical memory 114 for storing metadata, and the remaining physical memory 108 is for storing data. The inventors recognized that even if a compute node is one of compute nodes 102 with only one virtual machine using hardware security or other schemes, each compute node must reserve a range of physical memory 106 for storing metadata, resulting in an unacceptable memory overhead in the first approach. According to the first approach (left), the range of physical memory 106 reserved for storing metadata is allocated at compute node boot time and often reaches about 3% of physical memory. Assume a data center has multiple tenants, of which only 25% use hardware security or other schemes. Metadata allocation is not adjusted to match the (non-)use of schemes by tenants, but still, 25% of tenants using schemes and 25% of total data physical memory account for 100% of metadata allocation, thus imposing an effective memory overhead of 12-13%. Since memory is one of the most expensive resources in a data center, allocating 12-13% of physical memory is unacceptable.

[0020] According to the second approach (right side), and according to embodiments of the present disclosure, at boot time, by performing a fixed reservation, a small (not necessarily contiguous) range of the physical memory 110 is reserved for the metadata summary table. The metadata summary table is stored, in some cases, using 1 / 512 of the physical memory. The metadata summary table enables indirection (by storing pointers to locations within the physical memory storing the metadata) and, in some cases, enables a caching function (at the location storing the metadata). The metadata summary table is a representation of the physical memory that holds information about which regions of the physical memory store metadata, which regions store data, and which regions are still unallocated. The metadata summary table itself stores some metadata in certain situations, as will be described in more detail below, and in this regard provides a kind of caching function for the metadata being stored. The data is stored in the remaining physical memory 112, and in addition, a portion of the remaining physical memory 112 is dynamically allocated for storing metadata. Pointers to the physical memory regions dynamically allocated for storing metadata are stored in the metadata summary table. Thus, the data and its associated metadata are not stored together in the physical memory, i.e., they are not spatially adjacent. Embodiments of the present technology relate to this second approach in which the metadata summary table is used. The metadata summary table is much smaller than the range of the physical memory 106 of the first approach, so memory is significantly saved compared to the first approach. The metadata summary table enables indirection by storing pointers to locations within the physical memory storing the metadata, so that even when the metadata summary table is much smaller than the range of the physical memory of the first approach, it can have sufficient memory for storing the metadata.

[0021] Introducing indirection using a metadata summary table can increase the computational load compared to the first approach. However, enabling some cache features can alleviate the burden. Depending on the situation, storing the metadata in the metadata summary table itself eliminates the need to search for pointers to the locations within the physical memory where the metadata already exists in the metadata summary table, thus reducing the computational burden. In the example, if the metadata is the same in a contiguous area of the physical memory, that metadata is held in the metadata summary table.

[0022] In the approach of FIG. 1, data and metadata are stored separately in physical memory. Other approaches include storing metadata in bands, i.e., storing metadata together with its associated data in physical memory. Doing so involves using physical memory specialized for directly supporting metadata. However, memory chips that directly support metadata are expensive and not introduced in many current data centers. The inventors also recognized that using a memory chip that directly supports metadata does not bring the advantages of local characteristics that can be utilized to improve performance, as will be described below. Some operations on metadata gain the benefit of being able to inspect the metadata separately from the data. To efficiently perform these using physical memory that stores metadata directly, the memory needs to be able to query not only the rows of data and metadata but also the columns of metadata.

[0023] FIG. 2 is a schematic diagram of a central processing unit 200, a cache line 202, virtual memory 206, and physical memory 210 of a computing device, where the physical memory 210 stores both data and metadata. FIG. 2 is background information included to assist in the understanding of the present disclosure.

[0024] Assume that an application running on a virtual machine on a computing device (APP in Figure 2) uses pages 212,214 of memory allocated in virtual memory address space 206. Pages 212,214 of virtual memory are mapped to pages 216,218 of pseudo-physical / guest physical memory 208. Pages 216,218 of pseudo-physical / guest physical memory 208 are then mapped to physical memory pages 220,222. To store metadata, physical memory pages 220,222 are slightly expanded, as indicated by the shaded rectangle. The metadata is stored in physical memory 210, and the fact that address translation occurs does not affect the ability to discover the metadata. In other words, given a virtual memory address, conventional address translation methods can be used to translate the virtual memory address to a physical memory address, and the metadata stored in physical memory can be retrieved using the physical memory pages. When data and its associated metadata are retrieved from physical memory and placed into a cache line (as indicated by the arrow from physical memory page 220 to cache line chunk 204 in Figure 2), the data and its metadata flow together and are stored together in cache line 202. However, this type of approach does not work because there is usually not enough space in physical memory to store metadata together with the data. Therefore, embodiments of the present disclosure store the data and metadata separately in physical memory.

[0025] Figure 3 is background material and is included to aid in understanding this disclosure. Figure 3 is a schematic diagram of the computing device's central processing unit 200, cache line 202, virtual memory 206, and physical memory 210, where physical memory 210 stores data and metadata separately. Figure 3 illustrates one technique to avoid a fixed partition of physical memory for storing metadata. However, the approach in Figure 3 has drawbacks, as described below, which are addressed in this disclosure. By storing data and metadata separately, there is sufficient space for metadata. With respect to Figure 2, the application runs in a virtual machine of the computing device and uses virtual memory pages in virtual memory 206. In this example, the virtual memory pages include virtual memory page 300 for metadata only and virtual memory pages 302, 304 for data only. With respect to Figure 2, each page of virtual memory maps to a page in pseudo-physical / guest physical memory 208. With respect to Figure 2, each page in pseudo-physical / guest physical memory 208 maps to a page in physical memory 210. When a cache line fill operation occurs, the data and its associated metadata are retrieved from individual pages in physical memory 210 and stored in individual locations within cache line 202, as indicated by arrows 306 and 308 between physical memory 210 and cache line 202 in Figure 3. Since the data and its associated metadata are not stored together in cache line 202, subsequent operations on the data and its associated metadata are not straightforward. Therefore, embodiments of this technology enable the storage of data and metadata together in a CPU cache, such as cache line 202 or a hierarchical cache, even when the data and its associated metadata are stored separately in physical memory.

[0026] Figure 4 is a schematic diagram of a memory management unit 400 and a tag controller 406 of a computing device according to embodiments of the present disclosure. In various embodiments of the present disclosure, the function of updating and making available the metadata summary table is performed in the memory management unit 400 and / or the tag controller 406. The term “tag” is used to refer to a type of metadata, so “tag controller” is a function for controlling metadata. The tag controller does not necessarily have to operate in response to what the CPU is doing at that time. As the tag controller is part of the cache hierarchy, it performs cache line evicting or cache filling. The memory management unit 400 is part of the CPU 200. The CPU communicates with physical memory 210 through a cache that is a tiered cache comprising, in the example of Figure 4, a level 1 cache 402, a level 2 cache 404, further possible levels of cache (shown by dotted lines in Figure 4), and the tag controller 406. Additional cache layers may also exist between the tag controller 406 and the physical memory 210. The data and metadata are stored together in any layer of the cache between the tag controller 406 and the CPU 200, and individually in any layer between the tag controller 406 and the main memory 210. In embodiments of this disclosure, the CPU can operate as if the data and associated metadata were stored together in physical memory (when they are actually stored separately). This is achieved by using a tag controller such that there is a single address in the physical memory address space available to retrieve the data and its associated metadata. As described herein, by using a tag controller, there is a single physical address used to retrieve the data and metadata after address translation.

[0027] The processor, and most caches, are unaware of exactly how data and metadata are stored in physical memory.

[0028] In the example in Figure 4, the tag controller is located at the bottom of the cache hierarchy, immediately following physical memory 210. The advantage of having the tag controller immediately following physical memory includes simplification of implementation in some systems.

[0029] The computing device in Figure 4 has one or more memory controllers, which are not shown in Figure 4 for clarity. Since the memory controllers communicate directly with physical memory, in some embodiments the tag controller 406 resides within one of the memory controllers. In this case, if the tag controller 406 resides within one of the memory controllers, the tag controller 406 has a small cache within the tag controller to store metadata and, optionally, metadata summary table entries.

[0030] The advantage of having a tag controller inside the memory controller is that it provides a single point of integration.

[0031] In some embodiments, the tag controller is located immediately before the last level of cache, which stores data and metadata separately (unlike other cache levels where data and metadata are stored together). The advantage of this approach is that it allows for the dynamic trading of the amount of data and metadata stored in the general-purpose cache depending on the workload.

[0032] Figure 5 is a flowchart of the method performed by the memory management unit according to an embodiment of the present disclosure. The memory management unit has the functionality of a conventional memory management unit for translating between virtual memory addresses, pseudo / guest physical memory addresses, and physical memory addresses. In addition, the memory management unit is extended as described below. The memory management unit receives a memory access instruction as a result of an application running in a virtual machine running on the CPU, for example (500). The memory access instruction includes a virtual memory address. The memory management unit translates the virtual memory address to obtain a physical memory address (502). As part of the translation process, the memory management unit also obtains information which is permission information and / or memory type information. The translation information and permission are provided to the MMU by software, either by an explicit instruction or by software that maintains data structures such as page tables that the MMU can directly reference. The permission information indicates whether permission has been granted to store metadata associated with the physical memory address. The memory type information indicates whether the type of memory at the physical memory address is of a type to which metadata is potentially associated with the physical memory address.

[0033] The memory management unit (in operation 504) uses the information obtained in operation 502 to determine whether the memory access instruction permits metadata. Depending on whether the memory access instruction permits metadata, the memory management unit proceeds to operation 510 and resumes normal translation operations. Thus, when metadata is not used, such as when hardware security schemes or other schemes that use metadata are not required, the processing overhead is unaffected. This is very useful as it enables a zero-cost feature for virtual machines that do not use metadata storage. For computing devices with hardware that supports metadata and virtual machines that do not enable the use of metadata, address translation does not permit metadata.

[0034] (In operation 504) Depending on whether the memory access instruction allows metadata, the memory management unit proceeds to operation 506. In operation 506, the memory management unit checks whether the memory access instruction is compatible with the metadata. This check involves querying the metadata summary table. Depending on whether the memory access instruction is compatible with the metadata, the memory management unit proceeds to the normal translation operation 510. Because the memory access instruction is compatible with the metadata, the physical memory address has already associated with the metadata storage.

[0035] (In operation 506) a trap is sent to the system software (508) in response to a memory access instruction being incompatible with the metadata. A trap is a message or flag that indicates to the system software of a computing device that there is a fault. The fault is that the memory access instruction requires metadata storage space in physical memory, but this space is not currently allocated. The trap allows, enables, or triggers the system software to dynamically allocate space in physical memory to store the metadata for the memory access instruction. Having the memory management unit send the trap is beneficial because the memory management unit operates nearly synchronously with the central processing unit. In contrast, the tag controller is not synchronous with the central processing unit. Therefore, the inventors recognized that it is beneficial to use the memory management unit rather than the tag controller to generate the trap because generating the trap in the tag controller is likely to trigger more cache evacuation by the software attempting to allocate metadata space, making it very difficult to guarantee results.

[0036] Note that in Figure 5, there are two decision diamond symbols 504 and 506, which offer advantages in contrast to having a single decision diamond symbol. The process still works even if decision diamond symbol 504 is omitted. In the case of a single decision diamond symbol, it is inferred that metadata is permitted (i.e., decision diamond symbol 504) (compatibility check 506) from the fact that storage is already allocated for metadata. However, in this case, the metadata summary table needs to be known to the CPU, or at least to the MMU, which is problematic. Also, in the case of a single decision diamond symbol, there is not enough information available to distinguish between the various types of metadata schemes used in the data center. Assuming that some virtual machines in the data center use MTE only, some CHERI only, and some CHERI and MTE, the mere fact that some metadata storage is available is not sufficient to determine whether a particular type of metadata can be stored in a given page. Note that the term “page” refers to a unit of memory and is not necessarily the same as the unit of memory referred to as a page by the MMU.

[0037] Figure 5 illustrates how instructions executed in a computing device cause the memory management unit to perform translation operations in response to authorization information indicating that metadata is not permitted at a physical memory address.

[0038] Figure 5 illustrates how instructions in a computing device prompt the memory management unit to perform translation operations in response to confirming that the metadata summary is positive.

[0039] Figure 6 is a flowchart of a method performed by system software in a computing device according to embodiments of the present disclosure. A non-exhaustive list of examples of system software includes operating systems and hypervisors. The system software receives a trap as a result of operation 508 in Figure 5 (600). The system software identifies a faulty address associated with the trap by examining the information received in the trap (602). The faulty address is a physical memory address investigated as a result of a memory access instruction in Figure 5, which is found to have no associated metadata storage, even though the memory access instruction requires associated metadata storage. The system software identifies a location in physical memory that is available to be allocated for metadata storage (604). The system software invokes one or more instructions or calls the firmware of the computing device to configure the metadata storage (606). The system software may communicate explicitly or implicitly with system software in other trust domains during this process. During configuration, the microcode or firmware allocates the identified location in physical memory for metadata storage. This includes marking a new page for use as metadata storage or using a new slot in an existing page.

[0040] The microcode or firmware performs a check within a hardware security scheme, or any other scheme in which metadata is used, to determine whether a specified location is available for metadata storage for a specified data page.

[0041] The system software returns from the trap handler (608), and therefore the memory management unit recognizes that the trap has been handled. The MMU then retries the memory access instruction, i.e., repeats the process from operation 500 to Figure 5. This time, when the MMU reaches operation 506, the result is positive because metadata storage has been allocated, and the MMU can proceed to operation 510.

[0042] Figure 6 shows the system software, The memory management unit receives a trap, Identify the faulty address, which is a physical memory address. Identify a memory location to store metadata associated with a physical memory address, Using microcode or firmware, This diagram illustrates how the instructions for configuring a memory location identified for storing metadata are provided.

[0043] Figure 6 illustrates how the system software includes instructions to instruct the memory management unit to continue translation operations, depending on the normal configuration of the memory locations identified for storing metadata.

[0044] Figure 7 is a flowchart of a method according to an embodiment of the present disclosure. The method in Figure 7 is performed by a tag controller for cache line evicting, where the cache line is evicted to a lower-level cache or physical memory. If the cache line is evicted to a lower-level cache, that lower-level cache is a cache that stores data and metadata separately. As previously described, the tag controller is part of the cache hierarchy and is responsible for evicting cache lines from that part of the cache hierarchy and for cache filling to that part of the cache hierarchy. The tag controller optionally has its own internal cache, which is referred to herein as the tag controller's internal cache. If the cache line is evicted (700), the tag controller checks its internal cache to determine whether configured metadata storage exists for the location in physical memory or the lower-level cache where the cache line content is evicted (702). If the result from the internal cache is not conclusive, the tag controller queries the metadata summary table to determine whether configured metadata storage exists (704). If a tag controller has an internal cache, it optionally caches the query results in a metadata summary table within that internal cache.

[0045] Therefore, the tag controller can determine whether metadata storage is configured for where the cache line content is evicted (706). If metadata storage is not configured, the tag controller begins discarding the metadata in the evicted cache line (708).

[0046] Depending on the configuration of metadata storage in the determination diamond symbol 706, the tag controller checks whether the metadata in the evicted cache line has sufficient locality (710). Sufficient locality means that the metadata is the same in relatively close storage locations. A cache line consists of multiple consecutive data chunks, and (since both metadata and data are stored in the cache line) each chunk may have metadata items that are stored with the chunk in the cache line. Sufficient locality exists if the metadata is the same for multiple consecutive chunks. The tag controller examines the metadata of the chunks to check whether there are more consecutive chunks with the same metadata than a threshold number. The threshold is pre-set and varies depending on the amount of space available in the summary table.

[0047] If metadata locality in a cache line is insufficient, the tag controller writes the metadata to physical memory (714). Data from evicted cache lines is also written to physical memory. In physical memory, data and metadata are stored separately, so metadata is written to a different location in physical memory than the data. The tag controller optionally stores metadata associated with a cache line in its internal cache (or it needs to write the metadata to physical memory or a lower-level cache). Over time, metadata accumulates in the tag controller's internal cache, and when the internal cache is full, the tag controller writes the metadata back to physical memory. In this way, the frequency of writes to physical memory decreases.

[0048] If there is sufficient locality of metadata in the cache line, the tag controller writes the metadata to the metadata summary table in a compressed format (718). It is assumed that the metadata for each chunk in the cache line is the same. In this case, the metadata written to the metadata summary table is one instance of the metadata for a single chunk, indicating that the same metadata fits throughout the entire cache line. The compressed format of the metadata consists of the metadata values ​​and the range of memory locations to which the metadata values ​​fit. The tag controller optionally uses its internal cache to store the compressed metadata before writing it to the metadata summary table (716). Metadata caching and compression work to reduce access (both read and write) to metadata storage in physical memory. In various examples, there are caches for both the metadata summary table and the metadata itself.

[0049] The inventors recognized that sufficient locality verification in operation 710 of Figure 7 provides a useful performance advantage. Examples of locality properties for MTE and CHERI are given below. Both MTE and CHERI have the same basic request from the memory system and store out-of-band data transmitted through the cache hierarchy. MTE stores metadata using 4 bits for every 16-byte granule. CHERI uses 1 bit for every 16-byte granule to indicate whether the data is a valid function. Both forms of metadata have useful locality properties, as follows: • MTE metadata is the same for all granules within an allocation, so large allocations will have consecutive executions of the same metadata values. This is especially true for mapping large files where entire pages have the same metadata values. Because most programs contain large amounts of contiguous non-pointer data, the CHERI system can have long executions with invalid tag bit values. These can even cover an entire page.

[0050] Figure 7 illustrates the use of a tag controller, which is part of the computing device's cache hierarchy and is configured to take metadata into account when evicting and / or refilling cache lines of at least some of the cache hierarchy.

[0051] Boxes 700, 702, and 704 in Figure 7 show how the tag controller is configured as part of cache line evicting to determine whether metadata storage is configured for the physical memory address from which the cache line is evicted, by checking one or more of the tag controller's cache and metadata summary tables.

[0052] The negative result, indicated by the diamond symbol 706 in Figure 7, shows how the tag controller is configured to discard the metadata of the evicted cache line, in response to discovering that metadata storage is not configured.

[0053] The diamond symbol 710 in Figure 7 indicates how the tag controller is configured to check whether the metadata of an evicted cache line has sufficient locality, if sufficient locality is found, write the metadata of the evicted cache line to the metadata summary table, or write the metadata to physical memory if sufficient locality is not found.

[0054] Figure 8 is a flowchart of the method performed by the tag controller for cache filling according to an embodiment of the present disclosure. The cache to be filled is a cache that stores both metadata and data. The tag controller receives a cache fill request from a higher-level cache in the cache hierarchy (800). The cache fill request includes a physical memory address from which data is to be retrieved in order to fill the cache. The tag controller checks its own internal cache (if it has an available internal cache) to determine whether metadata storage configured for the physical memory address exists. If the result is inconclusive (i.e., the metadata is not found in the internal cache), the tag controller queries the metadata summary table and optionally caches the result in its internal cache (804). This is because the metadata summary table contains information about physical memory and information about whether metadata is configured for each unit of physical memory.

[0055] In the determination diamond symbol 806, the tag controller is in a position to determine whether metadata storage is already configured for the physical memory address (using the information obtained in operations 802 and 804). If it is not configured, the tag controller retrieves the data from physical memory (808). Next, the tag controller sets the metadata associated with the data in the cache to its default value (812). The cache stores both data and metadata. In this case, since the data has no associated metadata, the tag controller fills the metadata area for each chunk in the cache with the default value. Next, the tag controller fills the cache (810). If the cache supports independent tracking of the validity of data and metadata, the tag controller may provide the data and metadata as soon as they become available, rather than combining them into a single message.

[0056] If, in determination block 806, it is determined that metadata storage is configured for a physical memory address, the tag controller proceeds to determination block 814. In determination block 814, the tag controller examines the metadata summary table entry for the physical memory address to determine whether the metadata summary table entry contains sufficient information. If it contains sufficient information, the metadata summary table entry contains metadata. The tag controller then proceeds to retrieve the data from the physical memory address and add metadata to that data (816). Next, the tag controller fills the cache with the data and the added metadata (818).

[0057] In the determination diamond symbol 814, if the metadata summary table entry does not yet contain metadata, the metadata summary table entry provides a pointer to the location in physical memory where the metadata is stored. Thus, the tag controller retrieves the location in physical memory where the metadata is stored (820) and retrieves the metadata (822). The tag controller also retrieves the data from physical memory (824) (since the data and metadata are stored separately in physical memory), adds the metadata to the data, and populates the cache with the data and the added metadata (818).

[0058] Boxes 800, 802, and 804 in Figure 8 illustrate how the tag controller is configured as part of the cache fill of the cache hierarchy to determine whether metadata storage is configured for the physical memory addresses from which data is written to the cache hierarchy, by checking one or more of the tag controller's cache and metadata summary tables.

[0059] The negative result, indicated by the diamond symbol 806 in Figure 8, illustrates how the tag controller is configured to populate the cache hierarchy cache with data and metadata where the metadata is set to default values, in response to discovering that metadata storage is not configured.

[0060] The positive result, indicated by the diamond symbol 814 in Figure 8, illustrates how the tag controller is configured to check the metadata summary table, and then fill the cache hierarchy with the data and the associated metadata, in accordance with the discovery that metadata storage is configured and the relevant metadata is found in the metadata summary table.

[0061] The negative result, indicated by the diamond symbol 814 in Figure 8, illustrates how the tag controller is configured to check the metadata summary table if metadata storage is found to be configured, and to retrieve the location of the relevant metadata in physical memory from the metadata summary table if the relevant metadata is not found in the metadata summary table.

[0062] The method described above with reference to Figures 7 and 8 enables compression of long executions of the same metadata and provides a fast path, for example, when the metadata is uniform across the page. Caching such metadata executions within the tag controller eliminates the need for many reads to physical memory, and in fact, the system software can completely omit a more detailed metadata store for pages that guarantee uniform metadata. Using CHERI, conveniently compressing zeroed metadata executions is also useful for the system software to sweep memory for functions, without needing to retrieve metadata function data from memory for such executions.

[0063] The embodiments described herein enable dynamically allocated tiered metadata storage, thereby allowing storage to be tailored on a per-VM or per-application basis.

[0064] In some embodiments described herein, the computing device has a policy for metadata storage.

[0065] In the example, the policy for metadata storage is managed by the operating system kernel of the computing device in a non-virtualized system, by the hypervisor in a virtualized system, and by the hypervisor combined with privileged microcode or firmware in systems with ambiguous privilege boundaries and conventional virtualization. In embodiments where the computing device performs sensitive computing, the mechanism for metadata storage is implemented in the privileged firmware or microcode. If there is a sensitive computing system that removes the hypervisor from the trusted computing base for confidentiality or integrity reasons, the privileged firmware or microcode is configured so that no entity in the system can inspect or modify the metadata for a page, and in this case, the data cannot be modified either. In systems without support for sensitive computing, the operating system (OS) or other system software such as the hypervisor can be relied upon to maintain the same guarantee.

[0066] In embodiments where a computing device performs sensitive computing, the metadata summary table is managed by privileged firmware or microcode, and its layout does not need to be architecturally exposed outside of the privileged firmware or microcode. A simple mapping exists from any physical address to the corresponding entry in the metadata summary table. In systems providing sensitive computing support, the metadata summary table information used by the MMU may be stored in the same location as the reverse mapping or similar metadata for sensitive computing. Thus, no additional memory access for the MMU is required.

[0067] The cache and memory systems have a coherence point below which metadata and data are stored separately and then merged and flow into a single cache line. The tag controller is responsible for assembling data and metadata during load, splitting them during removal, and maintaining a cache of metadata values.

[0068] The embodiments described herein offer the advantage of "paying nothing for what you don't use." Large memory partitions exist only for VMs that use architectural features that require allocated physical memory, and the partitions are not larger than the value required by the specific feature requested.

[0069] The embodiments described herein offer the advantage that the (micro)architecture is as simple as possible in order to achieve high performance.

[0070] In examples using sensitive computing, the hypervisor, or microcode, or firmware is configured to ensure that metadata storage is not deleted from pages while metadata may still exist in the cache. When a metadata page is deleted from physical memory, the corresponding data page is also deleted, so the old data in the cache is safely discarded. It is the responsibility of the microcode or firmware to ensure that metadata pages are flushed from the cache before being returned to an untrusted hypervisor, just as it is when data pages are returned to an insecure memory area.

[0071] In examples using confidential computing, as described here, there is a software interface to privileged firmware or microcode. The software interface includes function calls that operate across privilege boundaries.

[0072] Privileged firmware or microcode exposes a few functions that manage the metadata summary table and the metadata it points to. The metadata summary table can be thought of as a contract between hardware and firmware, read by both the tag controller and the MMU, and maintained by the firmware. The tag controller can write to the metadata summary table and store metadata as the highest level in a hierarchical tag storage design. System software at the top of the firmware has a consistent interface to the firmware that does not expose the details of the implementation of the metadata summary table.

[0073] Each metadata page is logically treated as an array of metadata stores, with one metadata store per data page. The number of metadata stores in a single metadata page depends on the supported features. If a VM has metadata usage features that are disabled overall, the MMU can completely skip checking for metadata storage. This means that the hypervisor either provides metadata storage when each page is allocated to the VM, or allocates detailed metadata storage for the page lazily and dynamically after the VM has mapped it as a metadata page.

[0074] Hypervisors may also provide excess memory as needed. For example, a cloud provider might add 3.125% of the memory allocated to a regular VM with MTE enabled to a pool available for other VMs, which can then be reclaimed if metadata storage for those pages is desired. On a physical machine with 1 TiB of memory, if all VMs have enabled MTE but use MTE for only half of their pages, 16 GiB of memory remains and can be temporarily used for other VMs as long as it is available for reclaim by the hypervisor when a failure is detected.

[0075] In various embodiments, a computing device has a privileged boundary, and instructions executed on the side of the more trusted boundary expose multiple functions across the privileged boundary to manage one or more of the metadata summary tables and the metadata that the metadata summary tables point to.

[0076] In the example, one of several functions takes a physical page address and a set of metadata features as arguments, and returns true if the page can be used for metadata storage, and false otherwise, where the page is the same or a different unit of memory used by the memory management unit, or creates a mapping between a data page and a metadata storage slot so that the sensitive computing assurances of the host architecture are met, or decouples a data page from an associated metadata page.

[0077] In the example, one of several functions handles page transitions from metadata storage use to data storage use, ensuring that if a page crosses a privilege boundary, that page is not used for metadata storage.

[0078] Query for metadata size A particular implementation may require different amounts of storage due to different microarchitectural characteristics, which may exceed the amount of architecture state. In the example, some VMs in a data center use MTE, while others use CHERI. In this case, it is useful to have the ability to query the firmware or microcode to determine the amount of space required for metadata storage (as it will differ depending on whether MTE or CHERI is being used). In the example, this ability takes as an argument an instruction (e.g., MTE or CHERI) of the type of metadata to be stored for a given page.

[0079] Creating a metadata page In the example, privileged firmware or microcode exposes the ability to transfer a page from a data page to a metadata page. In the case of sensitive computing, the ability to create metadata pages is configured to keep data pages at an appropriate privilege level so that the sensitive computing is not compromised. This ability takes a physical page address as an argument and returns true or false depending on whether the physical page has been successfully transferred to the page to store metadata.

[0080] Physical memory is moved from the publicly available pool of physical memory using this feature before it becomes available for storing metadata.

[0081] In the case of confidential computing, the firmware checks whether the page passed as an argument to this function is owned by the caller's privilege level.

[0082] Allocate metadata storage for pages In this example, privileged firmware or microcode exposes the ability to allocate metadata storage for a page.

[0083] Each data page with associated metadata has an associated metadata storage slot.

[0084] A metadata storage slot is a unit of physical memory within a page for storing metadata. A function is used to create its mapping. In the example, this function takes the physical address of the data page as an argument. Another argument is the physical address of the page allocated to store the metadata. Another argument is the index or address of the available slots in the metadata page.

[0085] This function performs sufficient checks to enforce confidential computing guarantees on the host architecture. This function returns true if metadata storage has been successfully allocated to the page; otherwise, the function returns false. This function returns failure if the metadata or data page is not owned by the caller, if the metadata page is not a metadata page, or if the data page already has allocated metadata.

[0086] If no errors occur, the metadata summary table entry for the data page is updated to point to the specified slot in the given metadata page. The state of the metadata summary table associated with the metadata page is updated so that the metadata page contains at least enough metadata to prevent it from being released while still being referenced by the data page.

[0087] Deleting metadata storage from data pages In this example, privileged firmware or microcode exposes the ability to remove metadata storage from a data page.

[0088] Before a metadata page is reused for normal use, any data pages using that page to store metadata are decoupled using a function that removes metadata storage from the data page. The caller of this function removes the data page from the transformation table and invalidates any cache lines containing data from the data page. This ensures that no cache lines containing metadata exist above the tag controller.

[0089] The function to remove metadata storage from a data page takes the physical address of the data page as an argument and returns a value indicating success or failure. The function returns failure if the data page is not owned by the caller, is not a data page, or is a data page without assigned metadata.

[0090] Upon successful invocation of the function, the metadata summary table entry for the data page is reset to indicate that no metadata exists, and the recording status associated with the metadata page is updated to reflect the fact that this reference no longer exists. The firmware then invalidates any cache lines referencing this data page.

[0091] Querying metadata storage for data pages In this example, privileged firmware or microcode exposes the ability to query metadata storage for data pages.

[0092] This function takes the physical address of the data page as an argument and returns a value indicating success or failure.

[0093] This function returns failure if the data page is not owned by the caller, is not a data page, or does not have associated metadata.

[0094] Reusing metadata pages In this example, privileged firmware or microcode exposes functionality for reusing metadata pages.

[0095] Dynamic metadata storage is intended to allow system software to move memory between data usage and metadata usage, or vice versa. This feature handles page transitions from metadata storage usage to data storage usage.

[0096] This function takes the physical address of the metadata page as an argument and returns true if the page has been deleted, and false otherwise.

[0097] This feature performs thorough verification to enforce confidential computing assurances for the host architecture.

[0098] Figure 9 illustrates various components of an exemplary compute-based device 900, which can be implemented as any form of computing and / or electronic device, such as a mobile phone, data center compute node, desktop personal computer, or wearable computer, and in some examples, an embodiment of dynamically allocatable metadata storage is implemented.

[0099] The compute-based device 900 comprises one or more processors 910, which are microprocessors, controllers, or any other suitable type of processor, for processing computer executable instructions that control the operation of the device in order to dynamically allocate metadata storage. The processors have a memory management unit 914. In some examples, for example, when a system-on-chip architecture is used, the processor 910 includes one or more fixed-function blocks (also referred to as accelerators) that implement part of any of the methods shown in Figures 5 to 8 (rather than software or firmware). The cache hierarchy 916 comprises a tag controller (not shown in Figure 9). System software 904 is provided in the compute-based device, enabling application software 906 to run in the device.

[0100] Computer-executable instructions are provided using any computer-readable medium accessible by the computing-based device 900. Computer-readable mediums include, for example, computer storage media and communication media such as memory 902. Computer storage media such as memory 902 include volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, and program modules. Computer storage media include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media used to store information for access by computing devices. In contrast, communication media embody computer-readable instructions, data structures, program modules, etc., in modulated data signals such as carrier waves or other transfer mechanisms. As defined herein, computer storage media do not include communication media. Therefore, computer storage media should not be interpreted as propagating signals themselves. Although computer storage media (memory 902) are illustrated within the computing-based device 900, it will be understood that in some examples, storage may be distributed or located remotely and accessed via a network or other communication link (for example, using a communication interface 912).

[0101] The terms “computer” or “computation-based device” are used herein to refer to any device having processing capabilities such as executing instructions. Those skilled in the art will recognize that such processing capabilities are incorporated into many different devices, and therefore the terms “computer” and “computation-based device” respectively include personal computers (PCs), servers, mobile phones (including smartphones), tablet computers, set-top boxes, media players, game consoles, personal digital assistants, wearable computers, and many other devices.

[0102] The methods described herein are, in some examples, executed by machine-readable software on a tangible storage medium, such as a computer program comprising computer program code means adapted to perform one or all of the operations of the methods described herein when the program is executed on a computer, and the computer program can be embodied on a computer-readable medium. The software is suitable for execution on parallel or serial processors so that the operation of the methods can be executed in any suitable order or simultaneously.

[0103] Those skilled in the art will recognize that the storage devices used to store program instructions may optionally be distributed across a network. For example, a remote computer may store an example of the process described as software. A local computer or terminal computer may access the remote computer, download some or all of the software, and execute the program. Alternatively, the local computer may download some of the software as needed, or execute some software instructions on the local terminal and some on the remote computer (or computer network). Those skilled in the art will also recognize that, by utilizing conventional techniques known to those skilled in the art, all or some of the software instructions may be executed by dedicated circuits such as digital signal processors (DSPs) or programmable logic arrays.

[0104] As will be apparent to those skilled in the art, any range or device value given herein can be extended or modified without loss of the desired effect.

[0105] While the subject matter has been described using language specific to structural features and / or methodological behavior, it should be understood that the subject matter as defined in the attached claims is not necessarily limited to the specific features or behaviors described above. Rather, the specific features and behaviors described above are disclosed as exemplary forms of implementing the claims.

[0106] It will be understood that the benefits and advantages described above may relate to one embodiment or to several embodiments. The embodiments are not limited to solving some or all of the described problems, or having any or all of the described benefits and advantages. Furthermore, it will be understood that any reference to “one” refers to one or more of those items.

[0107] The operations of the methods described herein may be performed in any suitable order, or concurrently as necessary. In addition, individual blocks may be removed in any way without deviating from the scope of the subject matter described herein. A further example may be formed by combining any aspect of any of the other examples described above with any aspect of any aspect of any aspect described herein without losing the desired effect.

[0108] The term “equipped with” is used herein to mean that a specified method block or element includes, but such block or element does not constitute an exclusive list, and the method or apparatus may include additional blocks or elements.

[0109] The term “subset” is used herein to refer to a suitable subset of a set that does not contain all the elements of the set (i.e., at least one element of the set is missing from the subset).

[0110] The above description is given merely as an example, and it will be understood that various modifications can be made by those skilled in the art. The above specification, examples, and data provide a complete description of the structure and use of exemplary embodiments. Although various embodiments have been described above with some degree of specificity or by reference to one or more individual embodiments, those skilled in the art will be able to make many modifications to the disclosed embodiments without departing from the scope of this specification.

Claims

1. A computing device, A processor having a memory management unit, Memory that stores instructions and The instruction, when executed by the processor, is sent to the memory management unit. Receiving a memory access instruction that includes a virtual memory address, The process involves converting the virtual memory address to the physical memory address of the memory, and obtaining information associated with the physical memory address, wherein the information includes permission information and / or memory type information, the permission information indicating whether permission has been granted to store metadata associated with the data in the physical memory address, and the memory type information indicating whether the metadata is associated with the physical memory address. Determining that the acquired information indicates that the metadata is permitted to be associated with the physical memory address, Based on the determination that the metadata is permitted to be associated with the physical memory address, a query is performed on a metadata summary table stored in physical memory, wherein the metadata summary table enables indirection by storing a pointer to the location in physical memory where the metadata is stored. Based on querying the metadata summary table that stores the pointer, it is determined whether the metadata is compatible with the physical memory address based on the physical memory address configured to store the metadata, If the metadata is determined to be incompatible, a trap is sent to the system software of the computing device, the trap triggers a dynamic allocation of the physical memory to store the metadata associated with the physical memory address. The method involves using a tag controller, wherein the tag controller is part of the cache hierarchy of the computing device, and the tag controller is configured to perform cache line evicting and / or cache filling of at least a portion of the cache hierarchy based on the metadata. The tag controller is configured to determine, as part of a cache line evicting, that metadata storage is configured for the physical memory address from which the cache line is evicted by checking the tag controller's cache and / or the metadata summary table. The tag controller is configured to check whether the metadata of the cache line to be evicted is the same within a threshold number of consecutive chunks, and if the metadata is the same within a threshold number of consecutive chunks, write the metadata of the cache line to be evicted to the metadata summary table, and if the metadata is not the same within a threshold number of consecutive chunks, write the metadata to the physical memory.

2. The computing device according to claim 1, wherein the instruction causes the memory management unit to proceed with a conversion operation if the acquired information indicates that the metadata is not permitted at the physical memory address, or if the instruction determines that the metadata is compatible.

3. The memory stores system software to be executed in the processor, and the system software is The memory management unit receives the trap, Although the aforementioned memory access instruction requires associated metadata storage, a faulty address is identified, which is the physical memory address for which associated metadata storage is found to be absent. Identify a memory location for storing metadata associated with the aforementioned physical memory address, Using microcode or firmware, configure the identified memory location to store the metadata. A computing device according to claim 1, comprising instructions for the purpose of:

4. The computing device according to claim 3, wherein the system software includes instructions for instructing the memory management unit to continue the translation operation once the identified memory location for storing the metadata has been successfully configured.

5. The computing device according to claim 1, wherein the tag controller is either part of the memory controller of the computing device, located immediately before the last level cache of the cache hierarchy which stores data and metadata separately, or located between the last level cache of the cache hierarchy and the memory controller of the computing device.

6. The computing device according to claim 1, comprising a cache hierarchy, wherein the physical memory stores the data and the associated metadata separately, and the cache hierarchy stores the data and the associated metadata together.

7. The computing device according to claim 1, wherein the tag controller is configured to discard the metadata of the cache line being evicted when it discovers that the metadata storage is not configured.

8. The computing device according to claim 1, wherein the tag controller is configured to determine, as part of the cache fill of the cache hierarchy, whether metadata storage is configured for a physical memory address from which data is written to the cache hierarchy, by checking the cache of the tag controller and / or the metadata summary table.

9. The computing device according to claim 8, wherein the tag controller is configured to populate the cache of the cache hierarchy with the data and metadata set to default values ​​when it discovers that the metadata storage is not configured.

10. The computing device according to claim 8, wherein the tag controller is configured to check the metadata summary table when it discovers that the metadata storage is configured, and if relevant metadata is found in the metadata summary table, to fill the cache hierarchy with the data and the relevant metadata.

11. The computing device according to claim 8, wherein the tag controller, upon discovering that the metadata storage is configured, checks the metadata summary table, and if the relevant metadata is not found in the metadata summary table, uses a pointer to obtain the location in the physical memory where the relevant metadata is stored, the pointer being stored in the metadata summary table.

12. The computing device according to claim 1, comprising a privilege boundary, wherein instructions executed on the more trusted side of the privilege boundary expose multiple functions across the privilege boundary to manage the metadata summary table and / or the metadata pointed to by the metadata summary table.

13. One of the aforementioned functions takes a physical page address and a set of metadata features as arguments, and returns true if the page can be used for metadata storage, and false otherwise, where the page is the same as or different from the unit of memory used by the memory management unit, or To ensure that the confidential computing assurances of the host architecture are met, create a mapping between data pages and metadata storage slots, or Detach the data page from the associated metadata page. The computing device according to claim 12.

14. The computing device according to claim 12, wherein one of the aforementioned functions handles page transitions from metadata storage use to data storage use, and the page is transitioned across the privilege boundary when the page is not used for metadata storage.

15. It is a method, Using the processor's memory management unit, Receiving a memory access instruction that includes a virtual memory address, The virtual memory address is converted to the physical memory address of the physical memory, permission information and memory type information associated with the physical memory address are obtained, the permission information indicates whether permission has been granted to store metadata associated with the data in the physical memory address, and the memory type information indicates whether the metadata is associated with the physical memory address. The acquired permission information and memory type information are determined to indicate that the metadata is permitted to be associated with the physical memory address. By querying the metadata summary table stored in physical memory, it is determined whether the metadata is compatible with the physical memory address, based on the physical memory address configured to store the metadata, and the metadata summary table enables indirection by storing a pointer to the location in physical memory where the metadata is stored. If the metadata is determined to be incompatible, a trap is sent to the system software of the computing device, and the trap triggers a dynamic allocation of the physical memory to store the metadata associated with the physical memory address. The system includes using a tag controller, the tag controller being part of the cache hierarchy of the computing device, and the tag controller being configured to perform cache line evicting and / or cache filling of at least a portion of the cache hierarchy based on the metadata, The tag controller is configured to determine, as part of a cache line evicting, that metadata storage is configured for the physical memory address from which the cache line is evicted by checking the tag controller's cache and / or the metadata summary table. The tag controller is configured to check whether the metadata of the cache line to be evicted is the same within a threshold number of consecutive chunks, and if the metadata is the same within a threshold number of consecutive chunks, write the metadata of the cache line to be evicted to the metadata summary table, and if the metadata is not the same within a threshold number of consecutive chunks, write the metadata to the physical memory.

Images