System and method for inter-memory / intra-memory tearing

Abstract

To provide a system and a method for performing inter-memory and intra-memory tiering.SOLUTION: An inter-memory / intra-memory tearing system includes a host CPU and a memory module 110 including tier-1 memory, tier-2 memory, and tier-3 memory. The memory module performs inter-memory tiering between the tier-1 memory and the memory module, and performs intra-memory tiering between the tier-2 memory and the tier-3 memory.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates generally to memory tiering, and more particularly to systems and methods for inter / intra-memory tiering between a host system or embedded central processing unit (CPU) and a memory module, such as a Compute Express Link (CXL) Memory Module (CMM), to improve the computing power of the host system or embedded CPU by reducing the operational overhead associated with memory tiering. [Background technology]

[0002] Warehouse-scale computers (WSCs) are hierarchically organized systems with many processors that can exploit parallelism at the request and data levels. These systems form the core of cloud infrastructure and are part of the fundamental infrastructure that powers internet services used by billions of people every day, such as search engines, social networks, online maps, video sharing platforms, online shopping sites, and email services.

[0003] However, at a scale of several thousand nodes, cold data, which is infrequently accessed by each application, accounts for approximately 25% of the data on average. Placing such cold data in expensive memory such as dynamic random access memory (DRAM) increases the total cost of ownership (TCO). Therefore, to address this issue, memory tiering based on data access frequency has been used to optimize TCO.

[0004] More specifically, existing memory tiering techniques primarily focus on classifying data temperature into two categories, "hot" and "cold," and arranging data across two tiers of memory, e.g., DRAM and flash memory such as NAND flash. That is, hot data is preferably stored in more expensive DRAM, i.e., tier 1 memory, while cold data is preferably stored in cheaper NAND flash, i.e., tier 2 memory.

[0005] However, the types of in-memory data required in WSC are diverse and their size is constantly increasing. As a result, simply categorizing data temperature into two categories, hot and cold, and using two tiers of memory is not enough to optimize TCO. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] US Patent Application Publication No. 2004 / 0486946 Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention has been made in consideration of the above-mentioned conventional problems, and an object of the present invention is to provide a system and method for effectively allocating multiple data in tiered memory. [Means for solving the problem]

[0008] In order to achieve the above object, one aspect of the present invention provides a system for performing inter-memory / intra-memory tiering, comprising a host central processing unit (CPU), a memory module including a tier 1 memory, and a tier 2 memory and a tier 3 memory, the memory module being configured to perform inter-memory tiering between the tier 1 memory and the memory module, and to perform intra-memory tiering between the tier 2 memory and the tier 3 memory.

[0009] In order to achieve the above-mentioned object, one aspect of the present invention provides a method for performing inter-memory / intra-memory tiering using a memory module, comprising the steps of receiving first configuration information for inter-memory tiering between a tier 1 memory and the memory module from a host central processing unit (CPU), receiving second configuration information for intra-memory tiering between a tier 2 memory and a tier 3 memory from a CPU included in the memory module, tracking page meta of a memory page, and performing, for the memory page, either inter-memory tiering between the tier 1 memory and the memory module based on the first configuration information and the tracked page meta, or intra-memory tiering between the tier 2 memory and the tier 3 memory based on the second configuration information and the tracked page meta, wherein the tier 2 memory and the tier 3 memory are included in the memory module. [Effects of the Invention]

[0010] According to the present invention, it is possible to efficiently classify multiple data temperatures for various workloads and effectively place the data in tiered memory taking into account the characteristics of the tiered memory.

[0011] Furthermore, according to the present invention, multiple inter-memory / intra-memory tiering methods can be configured in parallel using multiple page tracking methods.

[0012] Furthermore, the present invention allows customization of the tiering technique taking into account the performance and endurance characteristics of the tiered memory. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an example of an inter-memory / intra-memory tiering system according to one embodiment. [Figure 2] FIG. 1 illustrates another example of an inter-memory / intra-memory tiering system according to one embodiment. [Figure 3] FIG. 1 illustrates a memory tiering module according to one embodiment. [Figure 4] 1 is a flowchart illustrating a method performed in an inter-memory / intra-memory tiering system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, specific examples of embodiments of the present invention will be described in detail with reference to the drawings.

[0015] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the disclosed aspects may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the subject matter disclosed herein.

[0016] The use of the phrase "one embodiment" or "an embodiment" throughout this specification means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment disclosed herein. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "according to one embodiment" (or other phrases of similar import) in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In this regard, the word "exemplary" as used herein means "serving as an example, instance, or illustration." An embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0017] Furthermore, particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Also, depending on the context of the discussion herein, singular terms may include the corresponding plural, and plural terms may include the corresponding singular. Similarly, hyphenated terms (e.g., "two-dimensional," "pre-determined," "pixel-specific," etc.) are sometimes used interchangeably with their non-hyphenated counterparts (e.g., "two-dimensional," "pre-determined," "pixel-specific," etc.), and capitalized terms (e.g., "Counter Clock," "Row Select," "PIXOUT," etc.) are sometimes used interchangeably with their non-capitalized counterparts (e.g., "Counter Clock," "Row Select," "PIXOUT," etc.). Such occasional interchangeable usages are not intended to be mutually exclusive.

[0018] Furthermore, please note that the various figures (including component diagrams) shown and discussed herein are for illustrative purposes only and are not drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals are repeated among the figures to indicate corresponding and / or analogous elements.

[0019] The terminology used herein is for the purpose of describing some example embodiments only and is not intended to limit the claimed subject matter. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0020] When an element or layer is referred to as being on, "connected to," or "coupled to" another element or layer, it can be directly on, connected to, or coupled to the other element or layer, or there may be intervening elements or layers present. In contrast, when an element is referred to as being "directly on," "directly connected to," or "directly coupled to" another element or layer, there are no intervening elements or layers present. Like numbers refer to the same elements throughout. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0021] As used herein, terms such as "first," "second," and the like are used as labels for the nouns that precede them and do not imply any kind of ordering (e.g., spatial, temporal, logical, etc.) unless expressly defined as such. Furthermore, the same reference numerals may be used across two or more figures to refer to parts, components, blocks, circuits, units, or modules that have the same or similar functionality. However, such usage is for simplicity of illustration and ease of discussion only and does not imply that the structural or architectural details of such components or units are the same across all embodiments or that such generically referenced parts / modules are the only way to implement some of the example embodiments disclosed herein.

[0022] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the subject matter belongs. Furthermore, terms as defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless expressly defined as such in this specification.

[0023] As used herein, the term "module" refers to any combination of software, firmware, and / or hardware configured to provide the functionality described herein in connection with the module. For example, software may be embodied as a software package, code, and / or instruction set or instructions, and the term "hardware" as used in any implementation described herein includes, for example, an assembly, hardwired circuitry, a programmable circuitry, a state machine circuitry, and / or firmware that stores instructions executed by the programmable circuitry, alone or in any combination. Modules may collectively or individually be embodied as circuitry that forms part of a larger system, such as, but not limited to, an integrated circuit (IC), a system-on-chip (SoC), an assembly, etc.

[0024] Various controllers, units, and blocks are referenced herein. Each of these controllers, units, and / or blocks is embodied by a processor executing a specific set of dedicated software instructions, such as a software module. The processor controls the operation of the controllers, units, and / or blocks by executing the instructions. Multiple controllers, units, and blocks are defined by a single common processor and different dedicated software instruction sets. The processor of any of the controllers, units, or blocks described herein is tangible and non-transitory. As used herein, the term "non-transitory" does not imply a permanent characteristic of a state, but should be interpreted as a characteristic of a state that persists for a certain period of time. The term "non-transitory" specifically excludes a specific carrier wave, signal, or other type of characteristic that exists only temporarily at any place or time. The processor is an article of manufacture and / or a mechanical component. The processor is configured to execute software instructions to perform the functions described in various embodiments herein. The processor may be a general-purpose processor or part of an application-specific integrated circuit (ASIC). The processor may be a microprocessor, microcomputer, processor chip, controller, microcontroller, digital signal processor (DSP), state machine, or programmable logic device. The processor may also be a logic circuit including a programmable gate array (PGA) such as a field programmable gate array (FPGA), or other types of circuits including discrete gates and / or transistor logic. The processor may be a central processing unit (CPU), a graphics processing unit (GPU), or both. Furthermore, the processor described herein may include multiple processors, parallel processors, or both. Multiple processors may be included in or coupled to a single device or multiple devices. The instruction set is read from a computer-readable recording medium.Furthermore, the instructions, when executed by a processor, are used to perform one or more of the methods and processes described herein. In particular embodiments, the instructions reside completely or at least partially within main memory, static memory, and / or the processor during execution.

[0025] Dedicated hardware implementations, such as ASICs, programmable logic arrays, or other hardware components, may be configured to implement one or more controllers, units, and / or blocks described herein. One or more embodiments described herein implement functionality using two or more specific hardware modules or devices interconnected to enable communication of associated control and data signals between and through these modules. Thus, the present invention encompasses software, firmware, and hardware implementations. Nothing described herein should be interpreted as being or being implemented or operable solely in software, without hardware such as a tangible, non-transitory processor and / or memory.

[0026] As mentioned above, traditional two-level memory tiering is insufficient to meet the current needs of WSC.

[0027] In particular, when one tiered memory is a long-latency memory such as flash memory, it is difficult to meet high Quality of Service (QoS) by only data temperature classification and page migration based on access frequency. Some prediction algorithms are required to prefetch high-temperature data early.

[0028] Furthermore, to consider the effective endurance of flash memory, swapped-out pages should be removed from solid-state devices (SSDs).Furthermore, it is necessary to control the write bandwidth by considering the endurance of flash memory.

[0029] Furthermore, the write amplification due to the unit difference between CXL (64 bytes) and SSD (4 kilobytes) should be significantly reduced.

[0030] Since data temperature depends on workload characteristics, the detection algorithm needs to be customized according to the workload characteristics and logical memory devices in the customer's main applications. For example, in one workload, hot data can be defined by access frequency, while in another application, hot data can be classified by recency information (information about the most recent access). Therefore, the data temperature detection algorithm needs to be dynamically controlled according to the workload characteristics.

[0031] FIG. 1 is a diagram illustrating an example of an inter-memory / intra-memory tiering system according to one embodiment.

[0032] 1, the system includes a host CPU 100, tier 1 memory 105 (e.g., DRAM), and a memory module 110 (e.g., CMM). The memory module 110 includes a memory tiering module 111, tier 2 memory 112 (e.g., DRAM), tier 3 memory 113 (e.g., NAND flash), and an embedded CPU 114.

[0033] Here, inter-memory tiering generally involves promoting hot pages from memory module 110 to tier 1 memory 105 and demoting warm pages from tier 1 memory 105 to memory module 110. Intra-memory tiering generally involves promoting warm pages from tier 3 memory 113 to tier 2 memory 112 and demoting cold pages from tier 2 memory 112 to tier 3 memory 113. During inter-memory / intra-memory tiering, memory module 110 tracks the access frequency of each memory page and classifies it as hot / warm / cold based on that information. The temperature level and reporting cycle for each page are defined by host CPU 100 or embedded CPU 114 and are repeatedly executed based on the operations defined by memory module 110.

[0034] More specifically, the memory module 110, i.e., the internal memory tiering module 111, provides the ability to operate independently on multiple tiering instances within it. For example, the host CPU 100 can request page tracking to promote hot data in the memory module 110 to the tier 1 memory 105. This corresponds to inter-memory tiering between the tier 1 memory 105 and the memory module 110. Because the host CPU 100 is not involved in the internal operations of the memory tiering module 111 related to the tier 2 memory 112 and the tier 3 memory 113, the promotion of hot pages to the tier 1 memory 105 should be possible regardless of their intra-memory tiering (i.e., memory tiering between the tier 2 memory 112 and the tier 3 memory 113). Conversely, the intra-memory tiering between the tier 2 memory 112 and the tier 3 memory 113 should be performed independently of the promotion of pages to the tier 1 memory 105, allowing the memory module 110 to perform multiple independent operations.

[0035] The host CPU 100 configures inter-memory tiering using a control application programming interface (API) via CXL.io or mailbox. When the host CPU 100 sends a master-to-series (M2S) request (e.g., MemRd, MemRdData, MemSpecRd, MemWr, etc.) to access the memory module 110 via CXL.mem, the memory tiering module 111 tracks the access frequency of each memory page according to its configuration. If the access frequency of a certain page exceeds a hot frequency threshold, the memory tiering module 111 reports the page as a hot page to the host CPU 100 (promoting the hot page). If the host CPU 100 has pre-configured the tier 1 memory 105 to promote hot pages, the memory tiering module 111 can directly move the hot page to the tier 1 memory 105, for example, via CXL.cache. For example, the host CPU 100 sends an M2S request to configure the tier 1 memory 105 to promote the hot page. However, the present embodiment is not limited to this. Furthermore, in order to effectively manage the free space in the tier 2 memory 112 and ensure the durability of the tier 3 memory 113, the host CPU 100 erases the promoted pages from the memory module 110.

[0036] Furthermore, the embedded CPU 114 can configure intra-memory tiering via a register. When the tier 2 memory 112 fills up beyond a configured resource-full watermark, the memory tiering module 111 attempts to demote some cold pages according to an eviction algorithm, such as least recently used (LRU), least frequently used (LFU), or adaptive replacement cache (ARC). Also, if some pages in the tier 2 memory 112 have not been accessed until they reach a cold age threshold, those pages are demoted to the tier 3 memory 113 (cold page demoting). In this case, the memory tiering module 111 controls the write bandwidth to ensure NAND durability. In contrast, if a page in the tier 3 memory 113 is accessed frequently beyond a warm frequency threshold, that page and its contiguous pages, for example, based on temporal or spatial locality, are directly promoted to the tier 2 memory 112 (warm page promotion) when the access frequency of the page exceeds the warm frequency threshold. For example, if page N is accessed above the warm frequency threshold, then consecutive pages N+1, N+2, etc. are also promoted directly to tier 2 memory 112.

[0037] 1, the above description assumes that access frequency is used as the page meta to be tracked, but the present invention is not limited to this. For example, other page meta, such as recency or file type, or a combination thereof, can be used to classify pages into hot, warm, and cold pages. For example, classification based on recency classifies pages by comparing the last access time of a page with a predetermined threshold or the access times of other pages. Classification based on file type classifies pages based on whether the page data corresponds to an executable file, a media file, or a temporary file.

[0038] Table 1 below shows an example of 13 APIs used in the inter-memory / intra-memory tiering technique that can be used by the host CPU 100 and / or the embedded CPU 114. Each API is independent for each memory tiering instance, and the instance is identified by a TIERING_ID.

[0039] [Table 1]

[0040] The multiple tiering instances shown in Figure 1 can distinguish or separate page tracking and page migration policies and provide algorithms to accommodate diverse workloads and applications. As a result, the parallel processing shown in Figure 1 allows the system to flexibly accommodate multiple data temperatures and multiple tiers of memory configurations.

[0041] Additionally, each tiering instance can perform page tracking via host-side profiling via CXL.cache or device-side profiling via CXL.mem.

[0042] FIG. 2 is a diagram illustrating another example of an inter-memory / intra-memory tiering system according to one embodiment.

[0043] 2, similar to Fig. 1, the system includes a host CPU 100, a tier 1 memory 105 (e.g., DRAM), and a memory module 210 (e.g., CMM). The components of the host CPU 100 and the tier 1 memory 105 have already been described with reference to Fig. 1, and therefore will not be described again here.

[0044] Memory module 210 includes memory tiering module 211, tier 2 memory 212 (e.g., DRAM), tier 3 memory 213 (e.g., NAND flash), embedded CPU 214, and tier 1 candidate memory 215 (e.g., static random access memory (SRAM)). Memory module 210, memory tiering module 211, tier 2 memory 212, tier 3 memory 213, and embedded CPU 214 operate similarly to memory module 110, memory tiering module 111, tier 2 memory 112, tier 3 memory 113, and embedded CPU 114 described above with reference to FIG.

[0045] The tier 1 candidate memory 215 is provided to enable removal of hot candidate pages from the tier 2 memory 212 and the tier 3 memory 213. More specifically, when a page is identified as a hot candidate page by the memory tiering module 211, for example using an access frequency histogram, the page is removed from the tier 2 memory 212 or the tier 3 memory 213 taking into consideration availability and durability. That is, after the page is removed from the tier 2 memory 212 or the tier 3 memory 213, it is temporarily stored in the tier 1 candidate memory 215 before being hot promoted to the tier 1 memory 105.

[0046] 3 is a diagram illustrating a memory tiering module according to one embodiment. For example, memory tiering module 111 of FIG. 1 and / or memory tiering module 211 of FIG. 2 may be configured as shown in FIG.

[0047] 3, the memory tiering module includes a CXL module 301, a page tracking module 302, a memory management module 303, a cache 304, an input / output (IO) scheduling module 305, and a memory-to-IO (M2I) mapping module 306. Each module in FIG. 3 is configured using hardware, software, or a combination thereof.

[0048] The CXL module 301 provides functions (CXL.io, CXL.mem, and CXL.cache) that comply with the CXL standard at the link layer / transport layer. For example, CXL.mem provides a function that allows the host to expand additional memory based on a load / store architecture.

[0049] The page tracking module 302 tracks meta (e.g., access frequency and / or recency) for each page. The tracking may be event-driven or may involve periodically scanning the page meta according to, for example, a design. For example, the page tracking module 302 scans page information such as page meta from the CXL module 301 via CXL.cache. However, the present embodiment is not limited to this.

[0050] The memory management module 303 controls page migration between tier 2 memory and tier 3 memory, for example, between the tier 2 memory 112 and the tier 3 memory 113. The memory management module 303 performs proactive cold page demotion (e.g., LRU, LFU, ARC), reactive warm page prefetching (based on temporal or spatial locality) in response to specific events (e.g., cache misses), and / or proactive hot page promotion (e.g., using access frequency histograms). For example, when a cache miss occurs, the missed page is loaded from tier 3 to tier 2, and a warm page predicted using a page access pattern based on temporal or spatial locality is also prefetched. For example, when a cache miss occurs, the memory management module 303 invalidates or evacuates data in the cache 304 based on the cache ID, writes the page where the cache miss occurred to the cache 304, and prefetches a warm page. However, the present embodiment is not limited to this.

[0051] The cache 304 manages data allocated in the tier 2 memory, and controls, for example, the allocation method (N-way set associative method or hash chain method).

[0052] The IO scheduling module 305 controls the operation of the SSD using a storage access and transport protocol such as NVMe (Non-Volatile Memory Express) and performs durability considerations (e.g., write control, page deletion, etc.).

[0053] The M2I mapping module 306 manages mapping information between memory and SSD. For example, the M2I mapping module 306 manages mapping between memory addresses and SSD addresses to reduce write amplification, and the IO scheduling module 305 throttles write bandwidth according to remaining endurance. For example, the mapping information between memory addresses and SSD addresses is called "memory-to-IO (M2I) mapping information." However, this embodiment is not limited thereto.

[0054] More specifically, to write data to an SSD, a memory address must be converted into an IO address of the SSD. The M2I mapping module 306 manages lifecycle information indicating how many times the entire capacity of the SSD has been written each time address conversion is performed. The IO scheduling module 305 adjusts the write bandwidth to meet the SSD's warranty conditions based on the remaining lifespan of the SSD. Generally, SSDs support one full drive write per day (DWPD). For example, if an SSD has a capacity of 4 TB, the maximum amount of data that can be written per day is limited to 4 TB to guarantee five years of use. Therefore, to maintain this warranty, the write bandwidth must be adjusted based on the remaining lifespan. For example, the IO scheduling module 305 adjusts the bandwidth taking durability into consideration by looking up and updating the M2I mapping information based on the remaining lifespan of the SSD. However, the present embodiment is not limited to this.

[0055] 4 is a flowchart illustrating a method performed in an intermemory / intramemory tiering system according to one embodiment. For example, the method illustrated in FIG. 4 is described below with reference to the intermemory / intramemory tiering system of FIG. 1. However, the method illustrated in FIG. 4 is not limited in this respect.

[0056] Referring to FIG. 4, in step 401, the memory module 110 receives first configuration information for inter-memory tiering between the tier 1 memory 105 and the memory module 110 from the host CPU 100.

[0057] In step 402, the memory module 110 receives second configuration information for intra-memory tiering between the tier 2 memory 112 and the tier 3 memory 113 also included in the memory module 110 from a CPU included in the memory module 110, i.e., the embedded CPU 114.

[0058] In step 403, the memory module 110 tracks the page meta of the memory page.

[0059] In step 404, the memory module 110 performs inter-memory tiering for the memory pages between the tier 1 memory 105 and the memory module 110 based on the first configuration information and the tracked page meta, or performs intra-memory tiering for the memory pages between the tier 2 memory 112 and the tier 3 memory 113 based on the second configuration information and the tracked page meta.

[0060] According to the above-described embodiments, by configuring multiple memory tiers in a memory module such as a CMM, it is possible to utilize inexpensive memory such as NAND to further optimize TCO.

[0061] Furthermore, the above-described embodiments improve the computing power of the host system (host CPU) or embedded CPU by reducing the operational overhead associated with memory tiering, such as page tracking for hot / cold classification.

[0062] Although the embodiments have been described above using the CXL memory standard as an example, the present invention is not limited to this, and various embodiments of the present invention can also be applied to other memory standards.

[0063] Embodiments and operations of the subject matter described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware (including the structures disclosed herein and their structural equivalents), or in one or more combinations thereof. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., as one or more modules of computer program instructions encoded on a computer storage medium for execution by or control of the operation of a data processing apparatus. Additionally or alternatively, the program instructions may be encoded in an artificially generated propagated signal, e.g., a mechanically generated electrical, optical, or electromagnetic signal. Such a propagated signal is generated for the purpose of encoding information for transmission to an appropriate receiving device for execution by a data processing apparatus. The computer storage medium may be, or may be included in, a computer-readable storage device, a computer-readable storage substrate, a random-access or serial-access memory array or device, or a combination thereof. Furthermore, the computer storage medium itself may be a source or sink of computer program instructions that are not propagated signals but are encoded in artificially generated propagated signals. A computer storage medium may also be, or be contained within, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices). Furthermore, the operations described herein may be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or data received from other sources.

[0064] While this specification may contain many specific implementation details, these implementation details should not be construed as limitations on the scope of the claimed subject matter, but rather as descriptions of features specific to particular embodiments. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, even if certain features are described as functioning in a particular combination or are originally claimed in such a combination, one or more features may be deleted from the claimed combination, such that the claimed combination may relate to a subcombination or variations thereof.

[0065] Similarly, although operations are depicted in the figures in a particular order, this should not be understood as requiring such operations to be performed in the particular order or sequential order depicted, or that all of the depicted operations be performed, to achieve desirable results. In some situations, multitasking or parallel processing may be advantageous. Furthermore, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems are typically integrated in a single software product or packaged into multiple software products.

[0066] Thus, specific embodiments of the subject matter have been described herein. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Moreover, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve desirable results. Certain implementations may benefit from multitasking or parallel processing.

[0067] Although the embodiments of the present invention have been described in detail above with reference to the drawings, the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the technical concept of the present invention. [Explanation of symbols]

[0068] 100 host CPU 105 Tier 1 Memory (DRAM) 110, 210 memory modules 111, 211 Memory Tiering Module 112, 212 Tier 2 Memory (DRAM) 113, 213 Tier 3 Memory (DRAM) 114, 214 Embedded CPU 215 Tier 1 Candidate Memory 301 CXL Module 302 Page Tracking Module 303 Memory Management Module 304 Cache 305 Input / Output (IO) Scheduling Module 306 Memory-to-IO (M2I) Mapping Module

Claims

1. 1. A system for performing inter-memory / intra-memory tiering, comprising: a host central processing unit (CPU); Tier 1 memory and a memory module including tier 2 memory and tier 3 memory; The memory module includes: performing inter-memory tiering between the tier 1 memory and the memory module; 10. A system configured to perform intra-memory tiering between the tier 2 memory and the tier 3 memory.

2. 2. The system of claim 1, wherein the inter-memory tiering between the tier 1 memory and the memory module includes at least one of promoting hot pages from the memory module to the tier 1 memory or demoting warm pages from the tier 1 memory to the memory module.

3. 2. The system of claim 1, wherein the intra-memory tiering between the tier 2 memory and the tier 3 memory includes at least one of promoting warm pages from the tier 3 memory to the tier 2 memory and demoting cold pages from the tier 2 memory to the tier 3 memory.

4. the memory module further includes a memory tiering module; The memory tiering module includes: receiving inter-memory tiering configuration information from the host CPU; Tracking page meta of memory pages based on the inter-memory tiering configuration information; 10. The system of claim 1, configured to classify the memory pages as either hot pages or warm pages based on the tracked page meta.

5. 5. The system of claim 4, wherein the memory tiering module is further configured to report the hot page to the host CPU if the memory page is classified as a hot page.

6. 5. The system of claim 4, wherein the page meta includes at least one of access frequency or recency.

7. The memory module includes: An embedded CPU; a memory tiering module, The memory tiering module includes: receiving intra-memory tiering configuration information from the embedded CPU; Tracking page meta of memory pages based on the intra-memory tiering configuration information; 10. The system of claim 1, configured to classify the memory pages as either hot pages, warm pages, or cold pages based on the tracked page meta.

8. 8. The system of claim 7, wherein the memory tiering module is further configured to report the hot page to the host CPU if the memory page is classified as a hot page.

9. 8. The system of claim 7, wherein the page meta includes at least one of access frequency or recency.

10. 8. The system of claim 7, wherein the memory tiering module is further configured to move the memory page from the tier 3 memory to the tier 2 memory if the memory page is classified as a warm page.

11. 8. The system of claim 7, wherein the memory tiering module is further configured to move the memory page from the tier 2 memory to the tier 3 memory if the memory page is classified as a cold page.

12. The memory tiering module includes: a page tracking module configured to track the page meta of the memory page; a memory management module configured to control page migration between the tier 2 memory and the tier 3 memory; a cache configured to manage data located in the tier 2 memory; an input / output (IO) scheduling module configured to perform durability considerations; 10. The system of claim 7, further comprising: a memory-to-IO (M2I) mapping module configured to manage mapping between memory addresses and solid-state device (SSD) addresses.

13. 10. The system of claim 1, wherein the memory module further comprises a tier 1 candidate memory.

14. The memory module includes: removing memory pages identified as hot pages from said tier 2 memory or said tier 3 memory; 14. The system of claim 13, further configured to temporarily store the memory page in the tier 1 candidate memory before the memory page is hot promoted to the tier 1 memory.

15. 14. The system of claim 13, wherein the tier 1 candidate memory comprises static random access memory (SRAM).

16. The system of claim 1 , wherein the memory module comprises a Compute Express Link (CXL) Memory Module (CMM).

17. 10. The system of claim 1, wherein the tier 3 memory comprises a NAND flash memory.

18. 10. The system of claim 1, wherein the tier 1 memory comprises dynamic random access memory (DRAM).

19. 10. The system of claim 1, wherein the tier 2 memory comprises dynamic random access memory (DRAM).

20. 1. A method for performing inter-memory / intra-memory tiering by a memory module, comprising: receiving, from a host central processing unit (CPU), first configuration information for inter-memory tiering between a tier 1 memory and the memory module; receiving second configuration information for intra-memory tiering between a tier 2 memory and a tier 3 memory from a CPU included in the memory module; tracking page meta for memory pages; and performing, for the memory pages, either inter-memory tiering between the tier 1 memory and the memory module based on the first configuration information and the tracked page meta, or intra-memory tiering between the tier 2 memory and the tier 3 memory based on the second configuration information and the tracked page meta; 10. The method of claim 9, wherein the tier 2 memory and the tier 3 memory are included in the memory module.

Images