Hybrid memory compression

Abstract

A hybrid memory device provides a two-level main memory hierarchy in a computer system. The hybrid memory device includes near memory (NM), which comprises one or more near memory modules (251-252; 1151-1152) controllable by one or more near memory controllers (241-242; 1141-1142). The hybrid memory device also includes far memory (FM), which comprises one or more far memory modules (253-254; 1153-1154) controllable by one or more far memory controllers (251-252; 1151-1152). The hybrid memory controller (260) of the hybrid memory device is configured to: monitor access requests to the far memory (FM) to identify bandwidth-demanding memory content in the far memory (FM); compress memory content in the near memory (NM) to form a dynamic cache from the memory space freed up by the compression; and accommodate at least a portion of the detected bandwidth-demanding memory content in the far memory (FM) in the dynamic cache in the near memory (NM). Advantageously, the metadata for the accommodation is stored in the far memory (FM), and the total memory capacity exposed by the hybrid memory device to the operating system (270; 1170) running on the computer system (200; 1100) is the sum of the capacities of the near memory (NM) and the far memory (FM).

Description

Technical Field

[0001] This disclosure generally relates to the field of data compression in memory of electronic computers. More specifically, this disclosure relates to a hybrid memory device that provides a two-level main memory hierarchy in a computer system, to a computer system including such a hybrid memory device and one or more processing units and one or more levels of cache, to a method for providing a two-level main memory hierarchy in a computer system, and to a hybrid memory controller for use in a hybrid memory device. Background Technology

[0002] Data compression is a general technique used to store and transmit data more efficiently by encoding frequently occurring datasets more efficiently than less frequent datasets. More efficient storage and transmission of data is generally a concern for a variety of reasons. In computer memory, such as main memory or cache memory, which holds data and computer instructions upon which the operation of processing devices depends, more efficient storage (say, K times) of said data is of interest because this could potentially reduce the size of said memory by K times, allowing data transfer between memory locations with potentially K times less communication capacity, and storing and transmitting said data within or between computer systems and / or between memory locations with potentially K times less energy consumption. Alternatively, K times more data can potentially be stored in available computer memory compared to the absence of data compression. This could be of great interest in achieving potentially K times higher computer performance without having to add more memory (which could be costly or simply due to resource constraints rather than desired). As another example, the size and weight of smartphones, tablets, laptops / desktops, or set-top boxes are limited because larger or heavier smartphones, tablets, laptops / desktops, or set-top boxes may have lower value to the end user; thus potentially reducing the market value of such products. However, larger memory capacity can potentially increase the market value of a product because larger memory capacity can lead to higher product performance, resulting in better product usability.

[0003] In summary, across the broader landscape of computerized products (both isolated and interconnected), data compression can potentially improve performance, reduce energy consumption, or decrease the cost and space required for memory. Therefore, data compression has broad applicability in a wide range of computerized products beyond those mentioned herein.

[0004] Dynamic Random Access Memory (DRAM) suffers from limited bandwidth. To alleviate this problem, heterogeneous memory systems, also known as hybrid memory (HM), which involve a two-level main memory hierarchy, are an attractive way to address this deficiency. The first level of hybrid memory is sometimes called near memory (NM), and the second level is sometimes called far memory (FM). In some implementations of hybrid memory, high-bandwidth memory (HBM) comprises the first level, while DRAM comprises the second level. HBM typically offers significantly higher bandwidth than DRAM, but HBM can be significantly more expensive than DRAM. Therefore, a hybrid memory implementation using both HBM and DRAM can provide a main memory with the same high bandwidth as HBM, the same capacity as DRAM, and almost the same low cost as DRAM.

[0005] Existing technologies have explored two broad approaches for managing hybrid memory: cache-type hybrid memory and flat-type hybrid memory. In cache-type hybrid memory, the NM (Nearest Memory) is used as a cache for the FM (Fast Memory), and is managed transparently to the operating system. However, for hybrid memory as considered in this patent disclosure, using HBM (Hybrid Basic Memory) and DRAM as the NM and FM respectively, the amount of DRAM is typically a small multiple, such as eight times, larger than the amount of HBM. Therefore, a significant amount of memory capacity is wasted by not exposing the NM to the (operating) system.

[0006] On the other hand, in flat hybrid memory, both the NM and FM contribute to the flattening of the physical address space, making the entire hybrid memory capacity available to the (operating) system. Here, bandwidth-demanding pages mapped in the FM (e.g., DRAM) are typically swapped with less bandwidth-demanding pages residing in the NM (e.g., HBM). Unfortunately, changing page mappings incurs significant operating system overhead and traffic overhead from swapping pages. To reduce the former type of overhead, existing technologies have proposed hardware-level remapping mechanisms that consider smaller granularities. However, the metadata required to track finer-grained access cells used for remapping can consume a significant portion of the NM capacity and may require significant on-chip memory resources to store the remapping metadata.

[0007] Existing technologies also consider an intermediate scheme between cache-type hybrid memory and flat hybrid memory, by statically reserving a portion of the NM (Nearest Medium) to cache data from the FM (Fast Moving Average) to avoid costly swapping operations. The remaining NM capacity is available to the system in flat mode. In one scheme, fine-grained swapping and caching are allowed. However, metadata still consumes valuable space in the NM. In another scheme, data is compressed in the NM to unknowingly expand its capacity for caching or flat space. However, cache space is still statically reserved from the flat space, and compression requires a staging area to stabilize the compressed data. Both of these lead to a reduction in the NM capacity available to the system. Summary of the Invention

[0008] As used herein, near memory (NM) and far memory (FM) are the first and second levels of a two-level computer main memory hierarchy, respectively. Typically, NM has higher bandwidth (or is faster or performs better in other ways), but may be more expensive than FM. We disclose an invention for hybrid memory compression. Unlike prior art, the disclosed method and system (i) expose the entire capacity of NM plus FM to the system in a flat mode, and (ii) dynamically expose a cache in NM from the capacity available for compressing data in NM. The freed cache is used to bring more bandwidth-demanding FM data into NM to avoid costly swapping operations. Metadata can be stored in FM through its novel management and carefully designed metadata layout. This disclosure imposes almost no area overhead in NM for metadata or for staging.

[0009] This disclosure unlocks space for caching by selectively compressing data in the NM. This is accomplished by dynamically assessing the compressibility and bandwidth requirements of fine-grained access units in the FM. By allowing fine-grained management of hybrid memory, it carefully manages compressed blocks in the NM (e.g., HBM) with minimal required metadata. For example, it compresses NM blocks at their original mapped locations and can cleverly use redundant ECC bits (in the NM, e.g., in the HBM) to locate compressed blocks, completely eliminating the use of remapping tables in the NM. As used herein, the term ECC bit refers to an error correction code bit.

[0010] Therefore, a first aspect of the invention is a hybrid memory device that provides a two-level main memory hierarchy in a computer system. The hybrid memory device includes a near memory comprising one or more near memory modules controllable by one or more near memory controllers. The hybrid memory device also includes a far memory comprising one or more far memory modules controllable by one or more far memory controllers. The hybrid memory controller of the hybrid memory device is configured to: monitor access requests to the far memory to identify bandwidth-demanding memory content in the far memory; compress memory content in the near memory to form a dynamic cache from the memory space freed by the compression; and accommodate at least a portion of the detected bandwidth-demanding memory content in the far memory in the dynamic cache in the near memory.

[0011] The hybrid memory controller can be configured to identify bandwidth-demanding memory content in the remote memory as memory content that is requested more frequently than other memory content in the remote memory. In some embodiments, the hybrid memory controller is configured to monitor access requests to the remote memory by incrementing a reference counter of the access unit of the remote memory each time access is requested, wherein the access unit is identified as bandwidth-demanding when the reference counter reaches or exceeds a threshold. In the general context of this disclosure, the term access unit refers to a memory page, a memory subpage, a memory superblock, or a memory block, wherein a memory page includes multiple memory subpages, a memory subpage includes multiple memory superblocks, and a memory superblock includes multiple memory blocks. Therefore, depending on the implementation, the hybrid memory controller can be configured to monitor access requests to access units that are memory pages, memory subpages, memory superblocks, or memory blocks.

[0012] The hybrid memory controller can be configured to: accommodate the access unit of the detected bandwidth-demanding memory content of the distant memory within a peer access unit of compressed memory content in the near memory, the peer access unit having the same relative starting address and the same size within a memory page addressable by an operating system running on the computer system as the access unit of the detected bandwidth-demanding memory content of the distant memory.

[0013] In the overall context of this disclosure, peering has the following meanings. A peer group is a set of memory pages in which the first N pages are mapped to near memory, and the subsequent K × N pages are mapped to far memory. Pages I <= N and pages I+N, I+2N, ..., I+KN in NM form peer groups. The size of the peer group is K+1. If two pages belong to the same peer group, then an access unit in the form of a memory page in the near memory is peered to an access unit in the form of another memory page in the far memory, and vice versa. If two access units belong to peer pages and have the same relative starting address within the memory pages, then an access unit in the near memory that is smaller than the memory page granularity is peered to an access unit of the same size in the far memory.

[0014] In some implementations, the hybrid memory controller is configured to manage the total memory space addressable by an operating system running on the computer system as follows: Subpages of memory pages are handled in three different modes: non-swappable mode, swapped mode, and cache / compressed mode.

[0015] For a subpage in non-swappable mode, the hybrid memory controller is configured to direct access requests to either the near memory or the far memory based on the page address mapping. If the subpage is located in the far memory, a reference counter for the subpage is incremented. If the reference counter reaches or exceeds a threshold, the subpage is switched to swap mode.

[0016] For a subpage in swap mode, the hybrid memory controller is configured to: upon entering swap mode, swap all access units of the subpage in the far memory with corresponding peer access units of the peer subpage in the near memory. For all access units of the subpage, the compressibility of each pair formed by the access units or sub-access units of the subpage in the far memory and the corresponding peer access units or sub-access units of the peer subpage in the near memory is examined, where sufficient compressibility means that the paired compressed access units or sub-access units will fit together in the peer access units or sub-access units of the peer subpage in the near memory. If the proportion of paired access units or sub-access units exhibiting sufficient compressibility in the subpage reaches or exceeds a compression threshold, the subpage is switched to cache / compression mode.

[0017] • For a subpage in cache / compressed mode, the hybrid memory controller is configured to: for each pair of access units or sub-access units in the subpage that exhibits sufficient compressibility, accommodate the compressed access unit or sub-access unit of the subpage in the far memory together with the corresponding compressed peer access unit or sub-access unit of the peer subpage in the near memory.

[0018] In some implementations, the hybrid memory controller is configured to store metadata for three different modes of subpages used to process memory pages in the remote memory.

[0019] In some implementations, metadata indicating the size of a compressed access cell or its sub-access cell in the far memory is encoded in unused control bits or unused portions of the peer access cell or its sub-access cell in the near memory. Such unused control bits may be, for example, ECC bits.

[0020] In some implementations, the hybrid memory controller is implemented on a microprocessor chip, which also includes one or more processing units, one or more levels of cache memory, the one or more near memory controllers, and the one or more far memory modules. The hybrid memory controller is operatively connected between the last level cache of the one or more levels of cache memory and the one or more near memory controllers and far memory modules.

[0021] In other embodiments, the hybrid memory controller is not a separate unit, but rather its functionality is integrated with one or more near-memory controllers.

[0022] The overall benefit of this disclosure is that it exposes the entire NM and FM capacity to the system. Advantageously, the metadata used for the containment (containing at least a portion of the detected bandwidth-demanding memory content of the far memory in the dynamic cache in the near memory) is stored in the far memory FM, and the total memory capacity (i.e., the total addressable memory space) exposed by the hybrid memory device to the operating system running on the computer system is the sum of the capacities of the near memory and the far memory. Data is compressed to free up NM capacity (e.g., HBM) to cache bandwidth-demanding blocks from the FM (e.g., DRAM). This includes techniques for dynamically assessing the compressibility and bandwidth requirements of fine-grained access cells in the FM.

[0023] Another overall benefit of this disclosure is its metadata layout, which keeps overhead low by (among other techniques) placing compressed blocks in the same locations as uncompressed blocks and using, for example, redundant ECC bits to store metadata in the NM compactly. This eliminates the need for any metadata in the NM and results in a modest on-chip memory structure for caching metadata from the FM.

[0024] A second aspect of the invention is a computer system comprising one or more processing units, one or more levels of cache memory, and a hybrid memory device. The hybrid memory device provides a two-level main memory hierarchy in the computer system, and includes a near memory comprising one or more near memory modules controllable by one or more near memory controllers. The hybrid memory device also includes a far memory comprising one or more far memory modules controllable by one or more far memory controllers. The hybrid memory controller of the hybrid memory device is configured to: monitor access requests to the far memory to identify bandwidth-demanding memory content in the far memory; compress memory content in the near memory to form a dynamic cache from the memory space freed by the compression; and accommodate at least a portion of the detected bandwidth-demanding memory content in the far memory in the dynamic cache in the near memory.

[0025] The hybrid memory device in the computer system according to the second aspect of the invention can be configured as defined for the hybrid memory device according to the first aspect of the invention, including any of its embodiments presented herein. Again, advantageously, the metadata for the containment (containing at least a portion of the detected bandwidth-demanding memory content of the far memory in the dynamic cache in the near memory) is stored in the far memory FM, and the total memory capacity (i.e., the total addressable memory space) exposed by the hybrid memory device to the operating system running on the computer system is the sum of the capacities of the near memory and the far memory.

[0026] A third aspect of the invention is a method for providing a two-level main memory hierarchy in a computer system. The two-level main memory hierarchy includes a near memory and a far memory, which together form a hybrid memory in the computer system. The method involves: monitoring access requests to the far memory to identify bandwidth-demanding memory content in the far memory. The method also involves: compressing memory content in the near memory to form a dynamic cache from the memory space freed by the compression; and accommodating at least a portion of the detected bandwidth-demanding memory content of the far memory in the dynamic cache in the near memory. Again, advantageously, the metadata used for accommodating (accommodating at least a portion of the detected bandwidth-demanding memory content of the far memory in the dynamic cache in the near memory) is stored in the far memory (FM), and the total memory capacity (i.e., the total addressable memory space) exposed by the hybrid memory device to the operating system running on the computer system is the sum of the capacities of the near memory and the far memory.

[0027] The method according to the third aspect of the invention may include the functionality defined by the hybrid memory device according to the first aspect of the invention, including any of its embodiments presented herein.

[0028] A fourth aspect of the invention is a hybrid memory controller for use in a hybrid memory device that provides a two-level main memory hierarchy in a computer system, the two-level main memory hierarchy including a near memory and a far memory, the near memory and the far memory together forming a hybrid memory in the computer system. The hybrid memory controller according to the fourth aspect of the invention is configured to: monitor access requests to the far memory to identify bandwidth-demanding memory content of the far memory; compress memory content in the near memory to form a dynamic cache from the memory space freed by the compression; and accommodate at least a portion of the detected bandwidth-demanding memory content of the far memory in the dynamic cache in the near memory. Again, advantageously, the metadata for accommodating (accommodating at least a portion of the detected bandwidth-demanding memory content of the far memory in the dynamic cache in the near memory) is stored in the far memory FM, and the total memory capacity (i.e., the total addressable memory space) exposed by the hybrid memory device to an operating system running on the computer system is the sum of the capacities of the near memory and the far memory.

[0029] The hybrid memory controller according to the fourth aspect of the invention may also be configured as the hybrid memory controller in the hybrid memory device according to the first aspect of the invention, including any of its embodiments presented herein.

[0030] This disclosure is not aware of the types of memory technologies considered for near-memory and far-memory. In one embodiment, high-bandwidth memory (HBM) can be used as near-memory, while dynamic random-access memory (DRAM) can be used as far-memory. In another embodiment, DRAM can be used as near-memory, and non-volatile memory (NVM) can be used as far-memory. In yet another embodiment, DRAM stacked on top of the microprocessor die can be used as near-memory, while DRAM located outside the microprocessor die can be used as far-memory. Other memory technologies for near-memory and far-memory can be considered by those skilled in the art, and all such embodiments are conceivable.

[0031] This disclosure is also agnostic regarding the selection of compression / decompression algorithms / accelerators, whether lossy or lossless. In one embodiment, base-delta immediate compression (BDI) can be used to compress pages, subpages, or superblocks mapped to distant memory together with pages, subpages, or superblocks mapped to near memory. In another embodiment, any entropy-based compression algorithm can be used for the same purpose. Those skilled in the art will consider other compression algorithms / accelerators for near and distant memory, and all such embodiments are conceivable.

[0032] The overall benefit of this disclosure is that the total memory capacity exposed by the hybrid memory device to the operating system running on the computer system is the sum of the capacities of the near memory and the far memory.

[0033] Other aspects, objects, features and advantages of the disclosed embodiments will become apparent from the following detailed patent disclosure, from the appended dependent claims and from the drawings.

[0034] Generally, all terms used in the claims should be interpreted according to their ordinary meaning in the technical field, unless otherwise expressly defined herein. All references to “a / an / the [element, device, component, means, step, etc.]” ​​should be openly interpreted as referring to at least one instance of the stated element, device, component, means, step, etc., unless otherwise expressly stated. The steps of any method disclosed herein need not be performed in the exact order disclosed, unless otherwise expressly stated. Attached Figure Description

[0035] Figure 1A computer system is described, the computer system including a microprocessor chip having one or more processing units, an exemplary three-level cache hierarchy, and one or more memory controllers for the near and far memories connected to one or more off-chip near and far memories.

[0036] Figure 2A Depicting and Figure 1 The same computer system is extended with a hybrid memory controller configured to manage the placement of data in far and near memory to allow data in near memory to be compressed to free up capacity for caching data in far memory.

[0037] Figure 2B Depicting Figure 2A An implementation of a hybrid memory controller, which includes a metadata cache, a control unit, a compression accelerator, and a decompression accelerator.

[0038] Figure 3 It describes how pages mapped to near and far memory form peer groups.

[0039] Figure 4 It describes how blocks in pages forming a peer group in both the far and near memory can be compressed together in the near memory, provided that the blocks are sufficiently compressible to allow data in the far memory to be cached in the near memory.

[0040] Figure 5 The metadata layout for handling the subpages and superblock sections of the page relative to each operating mode is described.

[0041] Figure 6A It describes how blocks in pages forming peer groups in far and near memory can be in three different operating modes: baseline mode, cache / compress mode, and swap mode.

[0042] Figure 6B It describes how peer blocks can be stored in near memory when they are sufficiently compressed to free up cache space for additional far memory blocks to be cached in near memory.

[0043] Figure 7 A flowchart is depicted showing a method for handling read / write requests for pages mapped to remote memory when the requested subpage is in swap mode.

[0044] Figure 8A flowchart is depicted showing a method for handling read / write requests for pages mapped to remote memory when the requested subpage is in cache / compressed mode.

[0045] Figure 9 A flowchart is depicted showing the method used to decide whether to cache or swap the superblock.

[0046] Figure 10 A flowchart is depicted for a method used to process remote memory write requests.

[0047] Figure 11 It is an alternative implementation method, wherein Figure 11 Depicting and Figure 2A The same computer system, but in which the hybrid memory controller is now integrated into each near-memory device.

[0048] Figure 12 An alternative implementation in which the hybrid memory controller is integrated into a high-bandwidth memory device is shown. Detailed Implementation

[0049] Figure 1 An exemplary implementation of a computer system 100 is depicted. The system includes a microprocessor chip 110 and two types of memory comprising an exemplary two-level hybrid memory hierarchy, wherein a plurality of M memory modules of one type (denoted as near memory (NM)) are shown as NM1151 to NM. M 152, and multiple L memory modules of another type (represented as far memory (FM)) are shown as FM1153 to FM L 154. The near memory (NM) and the far memory (FM) together constitute the hybrid memory (HM). The microprocessor chip 110 can be a discrete system employing any available technology or integrated on a system-on-a-chip (SoC). The microprocessor 110 includes components designated P1131, P2132, and so on. N 133 includes one or more processing units (sometimes referred to as CPUs or cores) and a memory hierarchy. In one exemplary embodiment, near memory is composed of high-bandwidth memory (HBM), while far memory is composed of dynamic random access memory (DRAM). However, near and far memory can be implemented in other combinations. For example, in one embodiment, near memory can be implemented using DRAM, while far memory can be implemented as non-volatile memory, such as phase-change memory (PCM) or magnetic RAM (MRAM).

[0050] exist Figure 1In this implementation, LLC requests are routed to either the HBM or DRAM controller, depending on whether the page is mapped to the NM or FM (in the preferred implementation, HBM and DRAM, respectively), as determined by the virtual / physical page address mapping.

[0051] On the other hand, the memory hierarchy includes multiple cache levels, such as... Figure 1 The example illustrates and is represented as three levels, C1, C2, and C3. These levels can be implemented using the same or different memory technologies, such as SRAM, DRAM, or any type of non-volatile memory technology, including, for example, spin-transfer torque random access memory (STT-RAM). The number of cache levels can vary in different embodiments, and exemplary embodiment 100 depicts three levels, with the last cache level being C3 120. These levels are connected using some kind of interconnection device (e.g., a bus or any other interconnection network). In the exemplary embodiment, levels C1 and C2 are related to P i The corresponding processing unit i (e.g., Figure 1 P1 in the table is private and can only be accessed by the entity represented as P. i The corresponding processing unit i (e.g., Figure 1 Access is made via P1 in the cache. As is well known to those skilled in the art, alternative implementations can have any number of private cache levels, or alternatively, all cache levels can be shared, such as... Figure 1 The third level, C3 120, is shown in the diagram. Regarding the inclusion of data within the cache hierarchy, any implementation is possible and will be understood by those skilled in the art. For example, C1 may be included within C2, while C2 may be non-inclusional relative to level C3 120. Alternative implementations will be understood by those skilled in the art.

[0052] Figure 1 Computer system 100 includes one or more memory controllers, wherein the memory controllers are of two types, having M near-memory controllers, denoted as NMCTRL1141 to NMCTRL M 142, and L remote memory controllers, denoted as FMCTRL1143 to FMCTRL L 144. The final cache level ( Figure 1 In this embodiment, C3 120 is connected to the memory controller, which in turn is connected to one or more memory modules of the corresponding type. In other embodiments, the final cache stage can be implemented as a library-based memory, where, for example, there is one library per processing unit. The memory controller can be integrated on the microprocessor chip 110, or it can be implemented externally to the microprocessor chip, for example, near the memory NM1151 and NM1151.M 152 are implemented closely together. The operating system (OS) 170 runs on the computer system 100, which is, of course, commonplace. Finally, the computer system 100 performs one or more tasks. These tasks can be any software application or part thereof run by the operating system 170 of the computer system 100.

[0053] According to Figure 1 In the first embodiment, represented as Baseline System 1 (BL1), the operating system 170 manages the two levels of the hybrid memory in a flat mode, where flat mode means that the capacity of each level in both levels is exposed to the operating system 170, so the total capacity is the sum of the NM and FM capacities. In this embodiment, the concept of peer groups, known from the prior art, is used to interleave pages, such as... Figure 3 As shown. Here, the page size is N, and page A 312 occupies addresses 0314 to N-1 316 and is mapped to NM, while pages B 322, C 332, D 342, and E 352 are mapped to FM. Assume a peer group has one NM page and four FM pages. For the meaning of peer, refer to the Summary of the Invention section. A recap follows. Assume that for a given set of memory pages, the first N pages are mapped to NM, and the subsequent K × N pages are mapped to FM. Then pages I <= N in NM and pages I+N, I+2N, ..., I+KN form peer groups. Only one page in an FM belonging to a peer group can be swapped with a page in an NM belonging to the same peer group. (Swapping will be explained in more detail later). We say the size of the peer group is K+1. As will be readily understood by those skilled in the art, this is similar to a direct-mapped cache of size N and a memory size of KN. Therefore, K blocks in memory are mapped to the same frame in the cache, and at any given time only one of the blocks can be located in the directly mapped cache.

[0054] The second implementation, denoted as Baseline System 2 (BL2), is built on top of BL1. When a page (or a portion thereof) mapped to the FM is deemed bandwidth-demanding, it is remapped to the NM, transparent to the operating system. This involves swapping with a corresponding page (or a portion thereof) located in the NM. For example, block K in page C 332 of the FM is equivalent to block K in page A 312 of the FM, and these two blocks will be swapped (see...). Figure 3 ).

[0055] The granularity chosen for the portion of a page used to monitor bandwidth requirements will determine the amount of metadata required to track it. The finer the granularity, the more metadata is needed. This will drive towards larger granular sizes. On the other hand, excessively large granules can lead to over-fetching of data due to limited spatial locality, resulting in excessive traffic overhead for exchanges. Therefore, the trade-off in choosing the granular size is between metadata overhead and spatial locality, and granular sizes other than pages and blocks will be considered in this patent disclosure. In the remainder of this document, and for the purpose of clarifying terminology, we will treat access units of the following granular sizes: page, subpage, superblock, and block, where the size is page=<subpage=<superblock=<block. As exemplary sizes for these access units, unless otherwise stated, we will consider 4 KB, 2 KB, 512 B, and 64 B for page, subpage, superblock, and block, respectively, where B refers to bytes.

[0056] The goal of the disclosed system and method is to free up capacity in the NM using a sufficiently fast compression technique and to use the freed capacity to cache bandwidth-demanding FM blocks. Without loss of generality, the NM can be exemplarily an HBM device, and the FM can be exemplarily a DRAM device.

[0057] As in the BL1 and BL2 implementations, pages are mapped to NM and FM by the operating system using peer groups. Go to Figure 2A It shows the same as Figure 1 The same implementation method, but with an extended functional block, denoted as HMCTRL 260. Figure 2A The remaining components can be combined with Figure 1 The corresponding elements in the figures are the same or equivalent (having the same reference numerals to denote the same elements, such that...) Figure 2A The microprocessor chip 210, processing unit 231, and operating system 270 in the middle can be connected with Figure 1The microprocessor chip 110, processing unit 131, and operating system 170 are identical or equivalent (etc.). The HMCTRL 260 intercepts all C3 220 requests. Initially, the HMCTRL 260 employs a BL1 strategy for all FM pages, meaning they are not swapped from the outset. This is called non-swapping mode. However, when an FM-mapped page is accessed, it is tracked by a reference counter at the subpage granularity. The reference counter increments each time the subpage is accessed. As long as the reference counter is below a preset threshold, all superblocks of the tracked subpage will be accessed from the FM and will not be swapped. However, when the reference counter exceeds the preset threshold, we say that the subpage is bandwidth-demanding, and the subpage will switch to swapping mode. From this point onward, all accessed FM superblocks associated with the subpage in swapping mode will be swapped with the corresponding NM superblocks belonging to the same peer group. We will provide details below on how to configure the device and how to establish methods to achieve this.

[0058] Once a subpage switches to swap mode, an attempt is made to obtain cache space in the NM through compression. This is done by attempting to compress the FM superblock in swap mode together with its corresponding NM peer superblock. We note that this disclosure is agnostic regarding the choice of compression algorithm, and that any reasonably fast compression algorithm in the prior art can be used. This disclosure is agnostic regarding the type of memory technology considered for NM and FM. In one embodiment, Basic-Delta Instant Compression (BDI) can be used to compress FM-mapped pages, subpages, or superblocks together with NM-mapped pages, subpages, or superblocks. In another embodiment, any entropy-based compression algorithm can be used for the same purpose. Those skilled in the art will consider other compression algorithms / accelerators for NM and FM, and all such implementations are conceivable.

[0059] When an FM superblock is requested, the HMCTRL 260 will also request a corresponding peer NM superblock. Next, for each pair of blocks in the two superblocks, it will be determined whether the two blocks are sufficiently compressed, meaning they can be contained within, say, a 64-byte block frame. If a certain percentage of the blocks within the requested superblock are sufficiently compressed using the above definition, the corresponding subpage is considered to be in cache / compressed mode. All sufficiently compressible blocks within the superblock will be compressed. Incompressible FM-mapped blocks will remain uncompressed in the FM.

[0060] From this point on, an attempt will be made to compress all superblocks within the subpage to place them in the NM. HMCTRL 260 will then update the metadata table (explained in detail later) so that subsequent requests for remapped superblocks are destined for the NM, not the FM. Otherwise, the subpage will remain in swap mode, and the requested superblocks will be swapped with the corresponding peer superblocks in the NM, explained in detail later.

[0061] Figure 2B It was shown at 2000. Figure 2A The implementation of HMCTRL 260 is described below. HMCTRL 260 includes four functional blocks: a metadata cache 2100, a control unit 2200, a compression accelerator 2300, and a decompression accelerator 2400. As will be explained in detail later, in some implementations, the metadata table is stored entirely in FM, with portions cached in HMCTRL 260. The cache for metadata table entries is... Figure 2B As shown at 2100. A portion of the memory address is used to index into the metadata cache, and the remainder is used as a tag, just like any cache structure known in the art. This cache structure can be a direct-mapped cache at one end and a fully associative cache or a set-associative cache at the other. To allow superblocks within subpages to be compressed, a compression accelerator 2300 is available. Similarly, to allow compressed superblocks to be decompressed, a decompression accelerator 2400 is available. These accelerators can be designed to accelerate any known compression method, such as entropy coding, basic delta instantaneous compression, or other methods known in the art. Detailed operation of the HMCTRL will be explained later; the control unit implementing this operation is another block 2200 in the HMCTRL 260 (i.e., 2000).

[0062] To see how compressible blocks are compressed, Figure 4 The diagram illustrates three consecutive compressed blocks (blocks N-1, N, and N+1) from two peer pages, A 410 and B 420. Here, blocks N-1 422, N 424, and N+1 426 from page B in FM are compressed and stored together with their corresponding peer blocks N-1 412, N 414, and N+1 416 from page A in NM 410. If LLC subsequently requests block N from page B 424, HMCTRL 260 will reroute the request to NM, to the Nth block cached in page A, based on metadata. In cases where the FM block and its corresponding peer NM block cannot fit into a 64-B block frame, the FM block remains in FM, and HMCTRL 260 will verify the response from NM before forwarding the request to FM, as explained in detail below.

[0063] Metadata Layout To enable HMCTRL 260 to determine what action to take for each LLC request, it uses a metadata cache. Back to Figure 2A The C3 220 request will be routed to NM or FM.

[0064] To recap, the HMCTRL 260 initially operates the hybrid memory in a flat mode, where pages are mapped to NM or FM by the operating system 270. However, when the reference count of a referenced FM-mapped subpage exceeds a preset threshold, it will remap from FM to NM at a superblock granularity. From this point onward, the requested FM-mapped superblock belonging to that subpage will be swapped with the corresponding peer NM-mapped superblock.

[0065] exist Figure 5 The layout of the metadata table is shown in box 510. Starting at box 510, the metadata table associates one entry with each peer group: metadata peer group 0 512 and metadata peer group 1 514. Therefore, it has the same number of entries as the number of pages in the NM. The metadata table is preferably stored in the FM (which saves valuable resources from the NM), or alternatively, it can be stored in the NM, but the metadata of the metadata table is cached in [the cache]. Figure 2A and Figure 2B In the metadata cache of HMCTRL 260 / 2000 (metadata cache indicator in) Figure 2B (At position 2100 in the middle). The metadata entry for a peer group is constructed to track one of all FM pages belonging to a pair of peer groups. Thus, box 520 shows the metadata for an exemplary peer group 1, and assuming that each page has two subpages, the metadata entry for a peer group requires a 2-bit tag 522 to indicate one of the four exemplary tracked FM pages in the peer group and the two subpages belonging to the tracked page (e.g., 4 KB) – subpage 0 metadata 524 and subpage 1 metadata 526 (e.g., each 2 KB), where each subpage has 15 bits of metadata.

[0066] Box 530 shows the 15-bit metadata field for each subpage. On the left, a single bit (swap / compression) 531, along with the content of the reference counter 540, indicates whether the subpage is in non-swap mode, swap mode, or cache / compression mode. If the reference count is below a preset threshold, the subpage is in non-swap mode, and the request will be destined for NM or FM based on the virtual-to-physical address mapping. If the reference count is above the preset threshold, the swap / compression flag 531 indicates whether the subpage is in swap mode (flag is set) or cache / compression mode (flag is reset). Next, for each of the four superblocks (exemplary size 512 B) belonging to a subpage of an exemplary size of 2 KB, there is a valid bit, 532, 533, ..., 535. The valid superblock bit indicates that the FM-mapped superblock is swapped (swap / compression bit 531 is set) or compressed in NM along with its corresponding NM-mapped superblock (swap / compression bit 531 is cleared). There are also 4 dirty superblock bits: 536, 537, ..., 539. These dirty superblock bits are set whenever a write-back request occurs in cache / compressed mode. Finally, the superblock reference counter 540 uses 6 bits.

[0067] Metadata Cache To recap, we assume that the metadata table is stored in FM. Figure 2A The HMCTRL 260 is configured to use a metadata cache—in Figure 2B The image shows metadata cache 2100—used to cache the contents of the metadata table. In one implementation, each metadata cache entry (e.g., 64 B) contains 16 consecutive metadata entries, each 32 bits. Therefore, the exemplary metadata cache design uses the address index corresponding to the requested peer group, stripping the least significant 4 bits. Given a NM size of C = 2... c And the page size is P = 2 p There exists N = 2 c-p A peer group. A peer group can be obtained by stripping the most significant c-bit from the physical page number.

[0068] When the metadata cache is hit Figure 2A The HMCTRL 260 will update the reference counter and retrieve the requested corresponding metadata entry from the metadata cache. Conversely, if the metadata cache is missed, Figure 2AThe HMCTRL 260 will first evict an entry to make room for the requested metadata entry, and if necessary, write the evicted metadata entry back to the location where the metadata table is stored, such as in the FM. The missing metadata entry from the FM is then fetched into the metadata cache.

[0069] Near-memory metadata support Figure 6A The diagram illustrates the mapping of blocks N-1, N, and N+1 in the baseline across two pages (A 612 and B 614) in cache & compression mode (page A 622 and page B 624) and swap mode (page A 632 and page B 634). Here, pages A and B belong to the same peer group, and page A is mapped to NM, while page B is mapped to FM. To recap, the... Figure 2A C3 220 block requests intercepted by HMCTRL 260 will be destined for NM in three scenarios, see reference. Figure 6A .

[0070] The first case is when the block is mapped to NM in non-swap mode. This corresponds to Figure 6A The baseline is 610. The second case is in exchange mode, when... Figure 5 When the swap / compression bit 530 is set and the superblock valid bit is set, that is, the bit among 532, 533, ..., 535 corresponding to that superblock. This corresponds to... Figure 6A In page 630, all blocks in page A612 have been swapped with blocks in page 614. Finally, the third case is in cache / compression mode, when... Figure 5 When the swap / compress bit 530 in the block is reset and the superblock valid bit is set ( Figure 5 (The one in 532, 533, ..., 535 that corresponds to that superblock). This corresponds to... Figure 6A The cache / compression is 620.

[0071] In cache / compression mode, it's uncertain whether the requested block is in the NM because the block might not be sufficiently compressed. Therefore, block-level information must be maintained in the NM, regardless of whether the FM-mapped block is compressed along with the corresponding peer NM-mapped block. If not, the FM block is placed in the FM, and the request must be rerouted to the FM.

[0072] HBM devices can sometimes associate 16 ECC bits with each 32-B access unit. When NM-mapped and FM-mapped blocks in the same peer group are compressed to fit into a 64-B block frame (two 32-B access units), in one implementation, six unused ECC bits out of the 16 can be used to encode the validity and size of the compressed NM-mapped and FM-mapped blocks. If an FM-mapped block is not compressed, it is stored in the FM. This is recorded by setting all six ECC bits to zero. If FM and NM-mapped blocks are compressed, their compressed sizes can be recorded in the corresponding six unused ECC bits. For example, if the compressed size is 63 bytes, the six ECC bits can be encoded as '111111', and if the compressed size is 2 bytes, the six ECC bits can be encoded as '000010'.

[0073] like Figure 6B As shown, we consider three consecutive blocks in two peer pages 662 and 664, denoted as N-1, N, and N+1. Here, we assume that page 662 is mapped to NM, and page 664 is mapped to FM. Now assume that block N in the subpages of peer pages 662 and 664 is in cache / compressed mode. This means that if Figure 6B If two blocks 666 and 668, represented as N in two peer pages 662 and 664 respectively, are adapted into a 64-block frame, they will be compressed and placed in NM. This is due to... Figure 6B Arrows 666 and 668 are shown in the diagram. For these two blocks, the NM block is placed starting from the original address of the block frame, while the corresponding FM-mapped block is mapped to the end of the block frame. Pointers 676 and 678 refer to 6 ECC bits, which are actually interpreted as the position of the last byte of each compressed block.

[0074] Using these unused ECC bits to encode the validity and size of compressed NM and FM map blocks can reduce memory waste and lower metadata overhead. In cases where ECC bits cannot be used, an alternative implementation is to store the metadata required for compression—i.e., the size of the compressed block—as part of the unused portion of the block. The only metadata needed outside of NM is an indication of whether the block is compressed, one bit per block. Another alternative is to combine the unused ECC bits with metadata bits stored in memory, as can be implemented by those skilled in the art. In yet another alternative, memory bits reserved for a purpose but not used for said purpose can be reused by those skilled in the art for metadata bits. All such alternatives are conceivable.

[0075] Detailed instructions Let's examine it in detail now. Figure 2AOperation of HMCTRL 260. To recap, HMCTRL 260 initially sends LLC requests to NM or FM based on their virtual-to-physical mapping when it is in non-switched mode relative to the LLC request.

[0076] When the reference count exceeds a preset threshold, the subpage will change from non-switch mode to swap mode. Figure 9 A flowchart illustrating the transition of a subpage from swap mode to cache / compress mode is shown. For each subpage in swap mode, and for each FM superblock request for that subpage, the first action is to request the FM superblock and its corresponding peer NM superblock 910. All blocks in both superblocks are compressed in pairs 920. Pairs of compressed blocks that fit into a 64B block frame are successfully compressed. If the proportion of successfully compressed NM and FM block pairs fitting into the block frame is higher than the compression threshold (TH) 930, the valid bit of the superblock is set, and the subpage is in cache / compress mode 940. Otherwise, the subpage is set to swap mode 950. In both cases, the process then terminates 960.

[0077] Transaction flow of last-level cache requests We are now considering such Figure 7 and Figure 8 The subpage shown is in swap mode. Figure 7 ) and cache / compression mode ( Figure 8 In the case of ( ), the transaction flow associated with LLC read or write requests to FM-mapped pages. For example... Figure 7 As shown, in swap mode, upon receiving an FM read or write request 710, HMCTRL 260 first increments the reference counter of the referenced subpage 720. It then checks if the superblock already exists in the NM due to a previous swap operation. If the superblock validity bit is set 730, HMCTRL 260 forwards the request to the NM 750. If the validity bit is cleared and the subpage's reference counter is below a preset threshold 740, HMCTRL 260 swaps the superblock in the FM with the superblock in the NM 760. This applies to both read and write FM requests in swap mode. Returning to decision box 740, if the reference counter is above the preset threshold, the request is forwarded to the FM 770. In all cases, the action terminates at box 780.

[0078] When considering Figure 8In the medium-high-speed cache / compression mode, when a subpage read request transaction (805) occurs, HMCTRL 260 first increments the subpage's reference counter (810), and then verifies the validity of the superblock (815). If the superblock is valid, the request is forwarded to NM (820). The validity of the block is then checked in NM (835). This may involve checking the ECC bit (830) in some implementations. If the block is valid, a response (845) is returned to HMCTRL 260. If the block is invalid, the request is forwarded to FM (840).

[0079] It's possible that some FM blocks within a superblock are cached in a compressed NM, while other FM blocks remain in the FM. The latter can be applied to FM blocks that cannot be compressed together with their corresponding, compressed NM blocks and stored within a 64-B block frame. For this reason, the HMCTRL 260 will inspect the response from the NM, including the data and the associated ECC bit. If the ECC bit is non-zero, the FM block is compressed and cached, and the HMCTRL will respond with an LLC. However, if the ECC bit is zero, the HMCTRL will forward the request to the FM 840.

[0080] Returning to decision box 815, which checks the validity of the superblock, the next action in the case of an invalid superblock is to check the reference counter of the corresponding subpage 825. If it is higher than a preset threshold, the superblock will be in cache / compress mode, and all blocks in the superblock will attempt to be compressed along with their counterparts in FM 850. On the other hand, if the reference counter of the corresponding subpage is not higher than the preset threshold, the request will be forwarded to FM 855. Regardless of the test results in the decision box, the process will eventually end at box 860, where it terminates.

[0081] Figure 10 The procedure for handling FM write request hits on child pages in cache / compressed mode is shown. The problem here is that from the last level cache (e.g., Figure 2AThe block written back (C3 220) may change in size after compression and may no longer fit. First, a test 1010 is performed to check if the superblock is valid. If invalid, the block does not exist in NM, and the request is forwarded to FM 1050. If the superblock is valid, an attempt is made to compress the block together with the corresponding peer block 1020. Second, a test 1030 is performed to check if the size of the compressed write-back block and the size of the peer block can fit within a 64B block frame. If they fit, the block is written back, and the ECC bit is updated to reflect its new size. Simultaneously, the superblock dirty bit is set. However, if it exceeds the size limit but can still fit within a 64B block frame with the block mapped by NM, the block is written to NM 1040, and the process terminates 1060. Finally, if it cannot be adapted, the block will be forwarded to the FM 1050, and in implementations that utilize the unused ECC bits of the peer block in the NM, these bits will be reset to reflect their invalidity.

[0082] Other related operations When a page in a peer group is being tracked, it's possible that another page in the same peer group will be accessed. Access to other pages within the same peer group will be ignored as long as the first page is not in swap mode. However, when a preset threshold is exceeded and a page is about to enter swap mode, access to other pages within the peer group will decrement the reference counter. If it reaches zero, the page will no longer be considered bandwidth-demanding and will transition from cache / compressed or swap mode to non-swapped mode. This transition requires all superblocks that may have been migrated to NM in uncompressed or compressed mode to be moved back to FM uncompressed. We call this operation page consolidation.

[0083] Page consolidation is also required when the operating system 270 changes the page mapping. Then, typically, the TLB entry to that page must be invalidated (TLB knockdown), and all blocks from said page must also be evicted. Page consolidation performs the latter as described below.

[0084] The page metadata is queried. For each subpage and each superblock of that subpage, if the superblock is in swap mode in NM, it will be swapped with the superblock in FM. If it is cached and compressed in NM, it will be decompressed and written back if the superblock is dirty, or it will be silently evicted if the superblock is not dirty.

[0085] So far, it has been made public. Figure 2AIn one implementation, the hybrid memory controller HMCTRL 260 is integrated on the microprocessor chip 210. Alternatively, it may be valuable to avoid integrating any of the functions of this disclosure onto the microprocessor chip 210. Instead, it may be valuable to integrate the functions of the hybrid memory controller HMCTRL within a near-memory device. Hereinafter, we refer to Figure 11 This leads to the idea of ​​an alternative implementation method.

[0086] Figure 2A and Figure 11 The difference between the implementation methods is that, Figure 2A The hybrid memory controller 260 has been replaced by interconnect 1160, which connects the last-level cache C3 1120 to the near and far memory controllers 1141 to 1142 (NMCTRL1 to NMCTRL1) respectively. M (relative to 1143 to 1144 (FMCTRL1 and FMCTRL)) L The function of the hybrid memory controller HMCTRL is no longer like... Figure 2A Instead of a separate unit like the 260, it is integrated with the near-memory controllers 1141 to 1142. This can... Figure 11 I saw it in [the text]. Figure 11 In this implementation, the operating system 1170 will determine whether the page is mapped to a distant memory device or a near memory device, and the virtual-to-physical address translation will determine whether the request is destined for a near memory device or a distant memory device.

[0087] exist Figure 12 In the example, we illustrate an implementation using high-bandwidth memory (HBM) as a near-memory device. The HBM device includes multiple memory dies 1220, 1230, and 1240, for example... Figure 12 There are K of them. These can be stacked on top of each other and can be connected to the logic layer 1210 using multiple through-silicon vias (TSVs), for example... Figure 12 The N values ​​observed at positions 1260, 1270, and 1280. In an alternative implementation, Figure 2A The hybrid memory controller 260 can be integrated into the logic layer of all near-memory devices (which may be HBM devices, for example).

[0088] Those skilled in the art will recognize that the physical placement of the hybrid memory controller neither affects its functionality nor its operation, and that what has been described regarding how it is configured and operated applies to this alternative implementation.

[0089] Concluding remarks A hybrid memory device providing a two-level main memory hierarchy in a computer system 200; 1100 has been disclosed above. The hybrid memory device includes a near memory NM, which includes one or more near memory modules 251 to 252, 1151 to 1152 controllable by one or more near memory controllers 241 to 242, 1141 to 1142. The hybrid memory device also includes a far memory FM, which includes one or more far memory modules 253 to 254, 1153 to 1154 controllable by one or more far memory controllers 251 to 252, 1151 to 1152. The hybrid memory device further includes a hybrid memory controller 260 configured to: monitor access requests to the far memory FM to identify bandwidth-demanding memory content of the far memory FM; compress memory content in the near memory NM to form a dynamic cache from the memory space freed up by the compression; and accommodate at least a portion of the detected bandwidth-demanding memory content of the far memory FM in the dynamic cache in the near memory NM. The entire capacity of the NM and FM is available to the operating systems 270 and 1170. Devices using one or more NMs and one or more FMs are envisioned. This means that the total memory capacity exposed to the operating systems 270 and 1170 is the sum of the capacities of the NM and FM.

[0090] The hybrid memory device dynamically monitors the access frequency and compressibility of pages, subpages (part or all of a page), and superblocks (part or all of a subpage). Based on the access frequency and compressibility of pages, subpages, and superblocks, the hybrid memory device determines which pages, subpages, or superblocks allocated by the FM should be migrated from the FM to the NM, and which pages, subpages, or superblocks allocated by the FM should be compressed together with pages, subpages, or superblocks allocated by the NM to allow pages, subpages, and superblocks to reside in the NM without using any of its physical space. Furthermore, the hybrid memory device is configured and operated in a manner transparent to software (e.g., the operating system). The hybrid memory device compresses data stored in the NM and stores data from the FM (also in compressed form) in the created space. Essentially, it creates a dynamic cache using the space created through memory compression.

[0091] The thresholds used for dynamic monitoring and dynamic decision-making as described herein are exemplary, and those skilled in the art may use other thresholds depending on how the objective is achieved.

[0092] This disclosure also discloses a computer system 200, 1100, which includes one or more processing units 231 to 233, 1131 to 1133, one or more levels of cache memory C1 to C3, and a hybrid memory device. The hybrid memory device provides a two-level main memory hierarchy in the computer system 200, 1100, and includes a near memory NM, which includes one or more near memory modules 251 to 252, 1151 to 1152 controllable by one or more near memory controllers 241 to 242, 1141 to 1142. The hybrid memory device also includes a far memory FM, which includes one or more far memory modules 253 to 254, 1153 to 1154 controllable by one or more far memory controllers 251 to 252, 1151 to 1152. The hybrid memory device further includes a hybrid memory controller 260 configured to: monitor access requests to the far memory FM to identify bandwidth-demanding memory content of the far memory FM; compress memory content in the near memory NM to form a dynamic cache from the memory space freed up by the compression; and accommodate at least a portion of the detected bandwidth-demanding memory content of the far memory FM in the dynamic cache in the near memory NM.

[0093] Another disclosed method is a method for providing a two-level main memory hierarchy in computer systems 200 and 1100, wherein the two-level main memory hierarchy includes a near memory (NM) and a far memory (FM), the NM and the FM together forming a hybrid memory (HM) in the computer systems 200 and 1100. The method involves monitoring access requests to the far memory (FM) to identify bandwidth-demanding memory content of the far memory (FM). The method also involves compressing memory content in the near memory (NM) to form a dynamic cache from the memory space freed by the compression; and accommodating at least a portion of the detected bandwidth-demanding memory content of the far memory (FM) in the dynamic cache in the near memory (NM).

[0094] Finally, a hybrid memory controller 260 for use in a hybrid memory device providing a two-level main memory hierarchy in computer systems 200 and 1100, the two-level main memory hierarchy including a near memory (NM) and a far memory (FM), the near memory (NM) and the far memory (FM) together forming a hybrid memory (HM) in the computer systems 200 and 1100. The hybrid memory controller 260 is configured to: • Monitor access requests to the remote memory FM to identify bandwidth-demanding memory contents of the remote memory FM. • Compress the memory contents in the near memory NM to form a dynamic cache from the memory space freed up by the compression, and • At least a portion of the detected bandwidth-demanding memory content of the far memory FM is accommodated in the dynamic cache of the near memory NM.

Claims

1. A hybrid memory device providing a two-level main memory hierarchy in a computer system (200; 1100), the hybrid memory device comprising: Near memory (NM), the near memory including one or more near memory controllers (241-242); 1141-1142) controls one or more near memory modules (251-252; 1151-1152); Remote memory (FM), the remote memory comprising one or more remote memory modules (253-254; 1153-1154) controllable by one or more remote memory controllers (251-252; 1151-1152); and A hybrid memory controller (260) is configured to: monitor access requests to the far memory (FM) to identify bandwidth-demanding memory contents of the far memory (FM); The memory contents in the near memory (NM) are compressed to form a dynamic cache from the memory space freed up by the compression; And to accommodate at least a portion of the bandwidth-demanding memory contents of the detected far memory (FM) in the dynamic cache in the near memory (NM).

2. The hybrid memory device according to claim 1, wherein, The metadata used for the containment is stored in the far memory (FM), and the total memory capacity exposed by the hybrid memory device to the operating system (270; 1170) running on the computer system (200; 1100) is the sum of the capacities of the near memory (NM) and the far memory (FM).

3. The hybrid memory device according to claim 1 or 2, wherein, The hybrid memory controller (260) is configured to identify bandwidth-demanding memory content of the far memory (FM) as memory content that is requested to be accessed more frequently than other memory content of the far memory (FM).

4. The hybrid memory device according to any of the preceding claims, wherein, The hybrid memory controller (260) is configured to: monitor access requests to the remote memory (FM) by incrementing a reference counter of the access unit of the remote memory (FM) each time an access unit of the remote memory (FM) is requested to be accessed, the access unit being one of a memory page, a memory subpage, a memory superblock, and a memory block, and wherein the access unit is identified as bandwidth-demanding when the reference counter reaches or exceeds a threshold.

5. The hybrid memory device according to any of the preceding claims, wherein, The hybrid memory controller (260) is configured to: accommodate the access unit of the bandwidth-demanding memory content of the detected far memory (FM) in a peer access unit of compressed memory content in the near memory (NM), the peer access unit having the same relative starting address and the same size as the access unit of the bandwidth-demanding memory content of the detected far memory (FM) within a memory page addressable by an operating system (270; 1170) running on the computer system (200; 1100).

6. The hybrid memory device according to any of the preceding claims, wherein, The hybrid memory controller (260) is configured to manage the total memory space addressable by the operating system (270; 1170) running on the computer system (200; 1100) as follows: Subpages of memory pages are processed in three different modes: non-swappable mode, swapped mode, and cache / compressed mode; For subpages in non-switch mode: Based on the page address mapping, the access request is directed to the Near Memory (NM) or the Far Memory (FM). If the subpage is located in the far memory (FM), the reference counter for the subpage is incremented, and If the reference counter reaches or exceeds the threshold, the subpage is switched to swap mode; For subpages in swipe mode: After entering the swapping mode, all access units of the subpages in the far memory (FM) are swapped with the corresponding peer access units of the peer subpages in the near memory (NM). For all access units of the subpage, the compressibility of each pair formed by the access unit or sub-access unit of the subpage in the far memory (FM) and the corresponding peer access unit or sub-access unit of the peer subpage in the near memory (NM) is examined, wherein sufficient compressibility means that the paired compressed access unit or sub-access unit will fit into the peer access unit or sub-access unit of the peer subpage in the near memory (NM); If the proportion of paired access units or their sub-access units exhibiting sufficient compressibility in the subpage reaches or exceeds a compression threshold, the subpage is switched to cache / compression mode; and For subpages in cache / compression mode: For each pair of access units or their sub-access units that exhibit sufficient compressibility in the subpage, the compressed access unit or its sub-access unit of the subpage in the far memory (FM) is accommodated together with the corresponding compressed peer access unit or its sub-access unit of the peer subpage in the near memory (NM).

7. The hybrid memory device according to claim 6, wherein, The hybrid memory controller (260) is configured to store metadata for three different modes of subpages used to process memory pages in the far memory (FM).

8. The hybrid memory device according to claim 6 or 7, wherein, Metadata indicating the size of the compressed access unit or its sub-access unit in the far memory (FM) is encoded in the unused control bits or unused portions of the peer access unit or its sub-access unit in the peer subpage in the near memory (NM).

9. The hybrid memory device according to any of the preceding claims, wherein, The near memory (NM) is high-bandwidth memory (HBM), and the far memory (FM) is dynamic random access memory (DRAM).

10. The hybrid memory device according to any one of claims 1 to 8, wherein, The near memory (NM) is DRAM, and the far memory (FM) is non-volatile memory (NVM).

11. The hybrid memory device according to any one of claims 1 to 8, wherein, The near memory (NM) is DRAM stacked on top of the microprocessor die, and the far memory (FM) is DRAM located outside the microprocessor die.

12. The hybrid memory device according to any of the preceding claims, wherein, The hybrid memory controller (260) is implemented on a microprocessor chip (210), which also includes one or more processing units (231-233), one or more levels of cache memory (C1-C3), one or more near memory controllers (241-242), and one or more far memory modules (253-254). The hybrid memory controller (260; 1160-1170) is operatively connected between the last level cache (220) of the one or more levels of cache memory (C1-C3) and the one or more near memory controllers (241-242) and far memory modules (253-254).

13. The hybrid memory device according to any one of claims 1 to 11, wherein, The functionality of the hybrid memory controller is integrated with the one or more near memory controllers (1141-1142).

14. A computer system (200; 1100) comprising one or more processing units (231-233; 1131-1133), one or more levels of cache memory (C1-C3), and a hybrid memory device, the hybrid memory device providing a two-level main memory hierarchy in the computer system, and the hybrid memory device comprising: Near memory (NM), the near memory including one or more near memory controllers (241-242); 1141-1142) controls one or more near memory modules (251-252; 1151-1152); Remote memory (FM), the remote memory comprising one or more remote memory modules (253-254; 1153-1154) controllable by one or more remote memory controllers (251-252; 1151-1152); and A hybrid memory controller (260) is configured to: monitor access requests to the far memory (FM) to identify bandwidth-demanding memory contents of the far memory (FM); The memory contents in the near memory (NM) are compressed to form a dynamic cache from the memory space freed up by the compression; And to accommodate at least a portion of the bandwidth-demanding memory contents of the detected far memory (FM) in the dynamic cache in the near memory (NM).

15. The computer system (200; 1100) according to claim 14, wherein, The metadata used for the containment is stored in the far memory (FM), and the total memory capacity exposed by the hybrid memory device to the operating system (270; 1170) running on the computer system (200; 1100) is the sum of the capacities of the near memory (NM) and the far memory (FM).

16. The computer system (200; 1100) according to claim 14 or 15, wherein the computer system has a hybrid memory device according to any one of claims 2 to 13.

17. A method for providing a two-level main memory hierarchy in a computer system (200; 1100), the two-level main memory hierarchy comprising near memory (NM) and far memory (FM), the near memory (NM) and the far memory (FM) together forming a hybrid memory (HM) in the computer system (200; 1100), the method relating to: Monitor access requests to the remote memory (FM) to identify bandwidth-demanding memory contents of the remote memory (FM); The memory contents in the near memory (NM) are compressed to form a dynamic cache from the memory space freed up by the compression; as well as At least a portion of the bandwidth-demanding memory contents of the detected far memory (FM) will be accommodated in the dynamic cache in the near memory (NM).

18. The method according to claim 17, wherein, The metadata used for the containment is stored in the far memory (FM), and the total memory capacity exposed by the hybrid memory device to the operating system (270; 1170) running on the computer system (200; 1100) is the sum of the capacities of the near memory (NM) and the far memory (FM).

19. The method of claim 17 or 18, further comprising the functionality defined for any hybrid memory device according to any one of claims 2 to 13.

20. A hybrid memory controller (260) for use in a hybrid memory device that provides a two-level main memory hierarchy in a computer system (200; 1100), the two-level main memory hierarchy including near memory (NM) and far memory (FM), the near memory (NM) and the far memory (FM) together forming hybrid memory (HM) in the computer system (200; 1100). The hybrid memory controller (260) is configured to: Monitoring access requests to the remote memory (FM) to identify bandwidth-demanding memory contents of the remote memory (FM). The memory contents in the near memory (NM) are compressed to form a dynamic cache from the memory space freed up by the compression, and At least a portion of the bandwidth-demanding memory contents of the detected far memory (FM) will be accommodated in the dynamic cache in the near memory (NM).

21. The hybrid memory controller (260), wherein, The metadata used for the containment is stored in the far memory (FM), and the total memory capacity exposed by the hybrid memory device to the operating system (270; 1170) running on the computer system (200; 1100) is the sum of the capacities of the near memory (NM) and the far memory (FM).

22. The hybrid memory controller (260) according to claim 20 or 21, wherein the hybrid memory controller is further configured as a hybrid memory controller in any one of claims 2 to 13.

Images