Direct swap cache with zero line optimization
By analyzing metadata and generating synthetic data blocks, the problem of uneven use of memory resources in multi-tenant computing systems is solved, and efficient utilization and performance improvement of memory resources are achieved.
Patent Information
- Application Number
- CN202380028705.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-12
- Filing Date
- 2023-01-26
- Publication Date
- 2024-11-15
AI Technical Summary
In multi-tenant computing systems, the uneven use of memory resources by the host operating system results in the failure to effectively allocate memory resources, resulting in waste of resources and degradation of performance.
By receiving a request and analyzing the metadata associated with the data block, it is determined whether the data blocks in the near memory and the far memory exist, and when all zero data is found in the far memory, a synthetic data block is generated to replace it, thereby optimizing data access. Reduce access to remote memory and implement speculative exchanges to reflect the state of data blocks.
It improves the utilization rate of memory resources, reduces the number of accesses to remote memory, saves power and time, and improves system performance.
Smart Images

Figure CN118974712A_ABST
Abstract
Description
Background Art
[0001] Multiple users or tenants can share a system including a computing system and a communication system. The computing system can include a public cloud, a private cloud, or a hybrid cloud with both a public part and a private part. The public cloud includes a global network of servers that perform various functions, including storing and managing data, running applications, and delivering content or services (such as streaming video, supplying emails, providing office productivity software, or processing social media). Servers and other components can be located in data centers around the world. Although the public cloud provides services to the public through the Internet, private clouds or hybrid clouds can be used for transactions. Both private clouds and hybrid clouds also include a network of servers placed in a data center.
[0002] Multiple tenants can use computing resources, storage resources, and network resources associated with servers in the cloud. Computing resources, resource storage, and network resources can be supplied using a host operating system (OS) installed on a computing node (e.g., a server) in a data center. Each host OS can allow multiple computing entities (such as virtual machines) to access computing resources and memory resources associated with a corresponding computing node. Because of the uneven use of memory resources by the computing entities supported by the host OS, the amount of memory resources may not be effectively allocated. As an example, a host server may not utilize a large amount of memory. Summary of the invention
[0003] In one aspect, the present disclosure relates to a method for managing a system having near memory and far memory. The method may include receiving a request from a requestor to read a data block stored in the near memory or in the far memory. The method may also include analyzing a metadata portion associated with the data block, the metadata portion including both: (1) first information related to whether the near memory includes the data block or whether the far memory includes the data block and (2) second information related to whether a data portion associated with the data block is all zero. The method may also include, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zero, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requestor.
[0004] In another aspect, the present disclosure relates to a system having a near memory and a far memory. The system may include a near memory controller configured to receive a request from a requestor to read a data block stored in the near memory or the far memory. The system may also include a near memory controller further configured to analyze a metadata portion associated with the data block, the metadata portion including: (1) first information related to whether the near memory includes the data block or whether the far memory includes the data block and (2) second information related to whether a data portion associated with the data block is all zero. The system may also include a near memory controller further configured to, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zero, synthesize the data portion corresponding to the data block to generate a synthesized data portion and send the synthesized data portion to the requestor instead of retrieving the data portion from the far memory.
[0005] In yet another aspect, the present disclosure relates to a method for managing a system having near memory and far memory. The method may include receiving a request from a requestor to read a data block stored in the near memory or in the far memory. The method may also include analyzing a metadata portion associated with the data block, the metadata portion including: (1) first information regarding whether the near memory includes the data block or whether the far memory includes the data block and (2) second information regarding whether a data portion associated with the data block is all zeros.
[0006] The method may also include, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zeros, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requester. The method may also include performing a speculative swap by writing the data portion corresponding to the data block previously stored in the near memory to the far memory and updating the metadata portion stored in the near memory to reflect that the near memory includes the data portion corresponding to the data block previously stored in the far memory.
[0007] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The present disclosure is illustrated by way of example and is not limited by the accompanying drawings, in which like reference numerals indicate similar elements. Elements in the figures are illustrated for simplicity and clarity and are not necessarily drawn to scale.
[0009] Figure 1 is a block diagram of a system including a computing node including access to both near memory and far memory according to one example;
[0010] Figure 2 A block diagram of a far memory implemented as a pooled memory system is shown;
[0011] Figure 3 is a diagram illustrating a transaction flow associated with a read operation according to an example;
[0012] Figure 4 is a diagram illustrating a transaction flow associated with a write operation according to an example;
[0013] Figure 5 is a diagram illustrating a transaction flow associated with a write operation according to an example;
[0014] Figure 6 is a diagram illustrating a transaction flow associated with a write operation according to an example;
[0015] Figure 7 An example transaction flow associated with a read operation and a subsequent write operation with a speculative exchange is shown;
[0016] Figure 8 A block diagram illustrating an example system for implementing at least some methods of a direct swap cache with zero-line optimization;
[0017] Fig. 9 A data center for implementing a system for a direct-swap cache with zero-line optimization is shown;
[0018] Fig.10 A flowchart illustrating an example method for managing memory having near memory and far memory; and
[0019] Fig.11 A flow chart of another example method for managing memory having near memory and far memory is shown. DETAILED DESCRIPTION
[0020] The examples described in this disclosure are related to systems and methods for direct exchange caches with zero-line optimization. Certain examples are related to direct exchange caches with zero-line optimization for host operating systems (OS) in computing systems or multi-tenant computing systems. The multi-tenant computing system can be a public cloud, a private cloud, or a hybrid cloud. The public cloud includes a global network of servers that perform various functions, including storing data and managing data, running applications, and delivering content or services (such as streaming video, email, office productivity software, or social media). Servers and other components can be located in data centers around the world. Although public clouds provide services to the public on the Internet, transactions can use private clouds or hybrid clouds. Both private clouds and hybrid clouds also include a network of servers placed in a data center. Computing entities can be performed using the computing resources and memory resources of a data center. As used herein, the term "computing entity" covers but is not limited to any executable code (in the form of hardware, firmware, software, or any combination of the foregoing) that implements functions, virtual machines, applications, services, microservices, containers, or single kernels for serverless computing. Alternatively, the computing entity may be executed on hardware associated with an edge computing device, a resident server, or other type of system including a communication system (such as a base station (e.g., a 5G base station or a 6G base station)).
[0021] In accordance with the examples of the present disclosure, the host OS can access a combination of near memory (e.g., local DRAM) and far memory (e.g., an allocated portion of pooled memory). As an example, a computing node in a data center can be allocated a pooled memory exposed by a pooled memory system, which can then be made accessible to the host OS running on the computing node as far memory. Pooled memory is associated with a memory including any physical memory shared by a plurality of computing nodes. In addition, data / instructions associated with the host OS can be swapped from far memory to near memory and from near memory to far memory. In this arrangement, near memory (e.g., local memory) can be implemented using expensive memory, and far memory (e.g., pooled memory) can be implemented using cheaper memory. As an example, expensive memory can correspond to a double data rate (DDR) dynamic random access memory (DRAM) (e.g., DDR2 DRAM, DDR3 DRAM, DDR4 DRAM, or DDR5 DRAM) operating at a higher data rate, and cheaper memory can correspond to a DRAM (e.g., DRAM or DDR DRAM) operating at a lower data rate. Other cost differences may be a function of reliability, or other differences in quality associated with near memory versus far memory. As used herein, the terms "near memory" and "far memory" are to be viewed in relative terms. Thus, near memory includes any memory used to store any data or instructions evicted from the system-level cache(s), and far memory includes any memory used to store any data or instructions swapped out from near memory.
[0022] Figure 1 1 is a block diagram of a system 100 including a computing node 110, a computing node 140, and a computing node 170 coupled to a far memory system 180 according to one example. Each computing node may include computing resources and memory resources. As an example, computing node 110 may include a central processing unit (CPU) 112; computing node 140 may include CPU 142; and computing node 170 may include CPU 172. Although Figure 1Each computing node in is shown as having a single CPU, but each computing node may include additional CPUs and other devices, such as a graphics processor unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other devices. In addition, each computing node may include near memory, which may be organized as a memory module. As an example, computing node 110 may include near memory in the form of memory module 122, memory module 124, memory module 126, memory module 128, memory module 130, and memory module 132. Computing node 140 may include near memory in the form of memory module 152, memory module 154, memory module 156, memory module 158, memory module 160, and memory module 162. Computing node 170 may include near memory in the form of memory module 182, memory module 184, memory module 186, memory module 188, memory module 190, and memory module 192. Examples of such memory modules include, but are not limited to, dual in-line memory modules (DIMMs) or single in-line memory modules (SIMMs). The memory included in the module may be dynamic random access memory (DRAM), flash memory, static random access memory (SRAM), phase change memory, magnetic random access memory, or any other type of memory technology that can allow the memory to be used as near memory.
[0023] Continue to refer Figure 1 , each computing node may also include a system-level cache controller (SLCC) and an associated cache memory (e.g., a system-level cache (not shown)). As an example, computing node 110 may include system-level cache controller (SLCC) 116, computing node 140 may include system-level cache controller (SLCC) 146, and computing node 170 may include system-level cache controller (SLCC) 176. In addition, each computing node may also include one or more memory controllers. As an example, computing node 110 may include memory controller 118, computing node 140 may include memory controller 148, and computing node 170 may include memory controller 178. In the case where the memory module includes DDR DRAM, the memory controller included in such a node may be a double dynamic rate (DDR) DRAM controller.
[0024] Each computing node may be configured to execute multiple computing entities. In this example, computing node 110 may have a host OS 114 installed on the computing node; computing node 140 may have a host OS 144 installed on the computing node, and computing node 170 may have a host OS 174 installed on the computing node. Far memory system 180 may include a logical pooled memory that may include multiple memory modules. Although not described in Figure 1 As shown in FIG. 1 , the far memory system 180 may include a logical pooled memory controller (described later). Examples of such memory modules include, but are not limited to, dual in-line memory modules (DIMMs) or single in-line memory modules (SIMMs). The memory included in these modules may be dynamic random access memory (DRAM), flash memory, static random access memory (SRAM), phase change memory, magnetic random access memory, or any other type of memory technology that may allow the memory to be used as near memory.
[0025] Any host OS (e.g., host OS 114, host OS 144, or host OS 174) executed by any computing node (e.g., computing node 110, computing node 140, or computing node 170) can access at least a portion of the physical memory included as part of far memory system 180. When the computing node is powered on or as part of an allocation operation / deallocation operation, far memory system 180 can allocate a portion of the pooled memory to the computing node. The allocated portion can include one or more "slices" of memory, where a slice refers to any minimum granularity of a portion of memory managed by a pooled memory controller (e.g., a memory page or any other memory block aligned with a slice size). Any suitable slice size can be used, including 1GB slices, 2GB slices, 8GB slices, or any other suitable slice size. The pooled memory controller can allocate or revoke the allocation of slices to a computing node based on a dispatch policy / revocation policy associated with far memory system 180. As explained earlier, data / instructions associated with the host OS can be swapped from far memory into near memory and from near memory out to far memory. In this arrangement, near memory (eg, local memory) may be implemented using expensive memory, and far memory (eg, pooled memory) may be implemented using less expensive memory.
[0026] In one example, computing node 110, computing node 140, and computing node 170 may be part of a data center. As used in the present disclosure, the term data center may include, but is not limited to, some or all data centers owned by a cloud service provider, some or all data centers owned and operated by a cloud service provider, some or all data centers owned by a cloud service provider and operated by customers of the service provider, any other combination of data centers, a single data center, or even some clusters in a particular data center. In one example, each cluster may include multiple identical computing nodes. Thus, a cluster may include a computing node that includes a certain number of CPU cores and a certain amount of memory. Instead of computing nodes, other types of hardware such as edge computing devices, local servers, or other types of systems including communication systems such as base stations (e.g., 5G base stations or 6G base stations) may also be used. Although Figure 1 System 100 is shown as having a certain number of components arranged in a certain manner, including computing nodes and memory components, but system 100 may include additional components or fewer components arranged differently. As an example, far memory system 180 may be included as part of each computing node, rather than as a separate component. Figure 1 Shown as separate systems.
[0027] Figure 2 Shows the corresponding Figure 1 180 is a block diagram of an example far memory system 200 of the far memory system 180 shown in FIG. The far memory system 200 may include a far memory system controller for coupling to a computing node (e.g., Figure 1 The far memory system 200 may further include a plurality of pooled memory controllers and associated pooled memory modules. As an example, the far memory system 200 may include a far memory controller (FMC) 210, FMC 220, FMC 230, FMC 240, FMC 250, and FMC 260 coupled to the switch 202, such as Figure 220. Each of the FMCs 210, 220, 230, 240, 250, and 260 may also be coupled to a fabric manager 280. The FMC 210 may also be coupled to a memory module 212, a memory module 214, a memory module 216, and a memory module 218. The FMC 220 may also be coupled to a memory module 222, a memory module 224, a memory module 226, and a memory module 228. The FMC 230 may also be coupled to a memory module 232, a memory module 234, a memory module 236, and a memory module 238. The FMC 240 may also be coupled to a memory module 242, a memory module 244, a memory module 246, and a memory module 248. The FMC 250 may also be coupled to a memory module 252, a memory module 254, a memory module 256, and a memory module 258. FMC 260 may also be coupled to memory module 262, memory module 264, memory module 266, and memory module 268. Each memory module may be a dual in-line memory module (DIMM) or a single in-line memory module (SIMM).
[0028] Continue to refer Figure 2In one example, each of the far memory controllers may be implemented as a pooled memory controller that complies with the Compute Express Link (CXL) specification. In this example, each of the memory modules associated with the far memory system 200 may be configured as a type 3 CXL device. The fabric manager 280 may communicate with the data center control plane 290 via the bus 206. In one example, the fabric manager 280 may be implemented as a fabric manager that complies with the CXL specification. The control information received from the data center control plane 290 may include control information that specifies which memory slices from the memory pool are allocated to any particular computing node at a given time. In response to the control information, the fabric manager 280 may allocate memory slices from within the far memory to a particular computing node in a time-division multiplexed manner. In other words, a particular slice of memory may only be allocated to a particular computing node at a time, and the particular slice may not be allocated to any other computing node. As part of this example, transactions associated with the CXL.io protocol, which is a non-coherent I / O protocol based on PCIe, may be used to configure memory devices and to configure links between CPUs and memory modules included in the far memory system 200. The CXL.io protocol may also be used by CPUs associated with various computing nodes in device discovery, device enumeration, device error reporting, and device management. Alternatively, any other I / O protocol that supports such configuration transactions may also be used. Memory access to memory modules may be handled via transactions associated with the CXL.mem protocol, which is a memory access protocol that supports memory transactions. As an example, load instructions and store instructions associated with any CPU may be handled via the CXL.mem protocol. Alternatively, any other protocol that allows CPU load instructions / store instructions to be converted into read transactions / write transactions associated with memory modules included in the memory system 200 may also be used.
[0029] Each far memory controller (e.g., any FMC 210, FMC 220, FMC 230, FMC 240, FMC 250, and FMC 260) can maintain a segment table indicating different portions of far memory (e.g., implemented as pooled memory) that can be allocated / unallocated at any suitable granularity with respect to portion size. More generally, the far memory controller can maintain any suitable table representing available memory slices / allocated memory slices, indicating any relevant information about the slices (e.g., allocated status / unallocated status, ownership status indicating to which compute node the allocated slice is allocated, recency of usage information, recency of allocation information, host type belonging to the compute node to which the allocated slice is assigned, or other metadata). For example, for a 2TB memory pool, portions can be allocated / unallocated at a granularity of 1GB slices, e.g., there can be 2K (e.g., 2048) segments in the segment table indicating different 1GB slices. As an example, a segment in a segment table may include a 32-bit segment identifier that includes 8 bits indicating which host the portion is assigned to, a 1-bit value indicating whether the portion has ever been accessed, a 3-bit decoder mapping indicating a target address decoding scheme used to address data in the portion, and / or a 16-bit leaky bucket counter indicating a count value of recent accesses to the portion. For example, the segment table described above may include an 8KB region of SRAM of a pooled memory controller. The schema for the segment table described above is non-limiting, and the segment table may include any suitable data for tracking the assignment of memory. Although Figure 2 The far memory system 200 is shown as having a certain number of components arranged in a certain manner, including the pooled memory controller and the memory modules, but the far memory system 200 may include additional components or fewer components that are arranged differently. As an example, the switch 202 may not be an integral part of the far memory system 200, and instead, the switch may be a separate component. In addition, multiple switches may be used. Moreover, the fabric manager 280 may be shared with additional pooled memory controllers or fewer pooled memory controllers.
[0030] In order to Figure 1In the context of the system 100 of FIG. 1 , near memory must have a fixed ratio with far memory. In this example, it is assumed that the near memory has the same size as the far memory. This means that any access to a location in the near memory will operate in the manner of a direct exchange cache. Therefore, the access will first perform a lookup in the memory designated as the near memory. Consistent with the unoptimized direct exchange cache arrangement, any hit in the near memory will be directly served from the near memory (e.g., local memory), while a miss in the near memory will cause a swap operation between the corresponding far memory location and the near memory location. The swap operation (e.g., swapping data from a location in the far memory to a location in the near memory or swapping data from a location in the near memory to a location in the far memory) can be performed at the granularity level of a cache line. Therefore, in this example, a data block is equivalent to a cache line. However, in this example, each location can have only one of the two cache lines at a given time. The other cache line is presented in the far memory.
[0031] Each cache line may include a combination of a data portion (e.g., 512 bits) and a metadata portion (e.g., 128 bits). The data portion may include data representing user data or instructions executed by a computing node. The metadata portion may include data representing various attributes of the data in the data portion. The metadata portion may also include error checking bits and error correction bits or other suitable types of information. In addition, the metadata portion may include a tag with an appropriate number (multiple) of bits to distinguish the location of the cache line. Table 1 shows example metadata information that can be used to distinguish the locations of two cache lines (e.g., CL$A and CL$B) and to let the near memory controller know whether the current cache line or another cache line includes zeros.
[0032]
[0033]
[0034] Table 1
[0035] The table above assumes a fixed ratio of 1:1 between the swappable range of near memory and the swappable range of far memory. However, the present disclosure is not limited to using a fixed ratio of 1:1 between the swappable range of near memory and the swappable range of far memory. As an example, a ratio of 1:3 can be used. In such a case, additional tag bits may be required to encode information about the location of the cache line in terms of the memory region with the cache line. In addition, additional metadata information may be maintained for the SLCC and memory controller to track the state of the cache line. Table 2 below shows example metadata information that can be used to achieve this purpose. Utilizing Figures 3 to 6 The transaction flow shown in further explains the use of the information in Tables 1 and 2.
[0036]
[0037]
[0038] Table 2
[0039] Figure 3 3 is a diagram illustrating an example transaction flow 300 associated with a read operation. During a read operation, a CPU (e.g., Figure 1 Any CPU 112, CPU 142, or CPU 172) may issue a request that is sent by a memory controller (e.g., Figure 1 The home agent may be associated with any of the memory controllers 118, 148, and 178 of the memory controller 118 to read the data. In this example, there is first a read request from the home agent to read cache line CL$B. The home agent may be a last level cache controller (e.g., Figure 1 The home agent may ensure that if multiple cores associated with the CPU are requesting access to a cache line, the requests are handled by the same control logic (e.g., the home agent). A memory controller (e.g., a memory controller previously associated with the CPU) may be a home agent. Figure 1Any near memory controller described) examines the contents of the near memory. The metatag portion in this example is "00", indicating that the near memory includes the cache line CL$A that caused the miss. In this example, the metatag portion also indicates that another cache line (CL$B) in the far memory includes only zeros. As a result, the included bit (INCL) is indicated as a logical value of "0" and another bit (OTHER) is also indicated as a logical value of "0". It is noteworthy that in this case, instead of retrieving cache line CL$B from the far memory, the memory controller synthesizes the data portion corresponding to cache line CL$B to generate a synthesized data portion (all zeros), and sends the synthesized data portion and the metadata portion to the home agent. Advantageously, as a result, no access to the far memory is required, saving both power and time. In one example, the synthesized data portion (e.g., all zeros) corresponding to the data portion (e.g., 512 bits) can be retrieved by a near memory controller (e.g., Figure 1 Any memory controller 118, memory controller 148, and memory controller 178) is generated by taking 512 bits that are grounded and therefore effectively zero and providing the 512 bits in response to a read operation. Alternatively, the contents of a 512-bit register storing zeros may be provided.
[0040] Relative to Figure 3 The read operations described can be of two types: a load read or a store read. The memory controller can determine the type of read operation from the semantics associated with the read operation. As in one example, the opcodes used for the read operations can be different. In the case of a load read, the memory controller will not issue a speculative access; however, in the case of a store read, the memory controller can issue a speculative access. The speculative access can be a speculative read to far memory to perform a swap operation between near memory and far memory. This can advantageously allow the CPU to have a hit in near memory during a write back operation. With respect to Figure 7 Further describing speculative exchange.
[0041] Figure 4 4 is a diagram illustrating an example transaction flow 400 associated with a write operation. During a write operation, a CPU (e.g., Figure 1 Any CPU 112, CPU 142, or CPU 172) may issue a request that is sent by a memory controller (e.g., Figure 1 The example relates to the home agent writing cache line $B with XYZ data and is Figure 3. Thus, the memory controller communicates to the system level cache controller (SLCC) that the states of the included bit (INCL) and another bit (OTHER) are logic "0". In this example, first there is a read request from the memory controller to read cache line CL$B. With this read request, the memory controller (e.g., Figure 1 The memory controller (as used herein) examines the contents of the near memory. The metatag portion in this example is 00, indicating that the near memory includes the cache line CL$A that caused the miss. However, the metatag portion also indicates that another cache line (CL$B) in the far memory includes only zeros. In this case, instead of retrieving cache line CL$B from the far memory, the memory controller performs an automatic swap operation. The automatic swap operation includes the memory controller writing the XYZ data into the near memory for cache line CL$B and changing the metatag portion from 00 to 10, indicating that cache line CL$B does not include all zeros. Because the cache line has all zeros in both the near memory locations and the far memory locations, no changes are made with respect to the metatag portion (00) associated with cache line CL$A. Advantageously, as a result, no access to the far memory is required, saving power and time.
[0042] Figure 5 is a diagram showing a transaction flow 500 associated with a transaction using additional tag bits for tag information and using data stored for each cache line. Transaction flow 500 is Figure 4 . When a cache line (e.g., cache line CL$A) is first brought into the system-level cache, the included bit is set. Using the included bit, the system-level cache controller (e.g., Figure 1Any of SLCC 116, SLCC 146, and SLCC 176 tracks whether a cache line is guaranteed to be present in near memory. Thus, in the example above, when the SLC controller retrieves a cache line from near memory, the included bit for the cache line can be set. When the cache line is evicted from the system-level cache to be written to near memory, if the included bit is still set, this information is communicated to the near memory controller, and the near memory controller uses this information to skip the read before the write. Thus, as an example, when cache line CL$A is to be written to near memory, the system-level cache controller checks the included bit in the tag to determine whether the included bit is set. If the included bit is set, the system-level cache controller can notify the memory controller to skip the read operation from near memory before writing to the memory space in near memory corresponding to the cache line. While cache line CL$A is in the system-level cache, if the system-level cache controller receives a read request for any other cache line (e.g., cache line $C) in the same cache line set as cache line CL$A, the system-level cache controller will clear the included bit for all other cache lines in the same cache line set. Thus, in the system-level cache, the included bit for cache line CL$C is set, while the included bits for all other cache lines in the same set are turned off. This is the system-level cache controller's way of communicating to the memory controller that it can no longer guarantee that a cache line within that particular cache line set will still be present in near memory when the cache line is written to near memory.
[0043] Continue to refer Figure 5 , as indicated in transaction flow 500, in response to a read request to read data for cache line CL$B, there is a hit in the near memory. As a result, data (e.g., XYZ) from the near memory for cache line CL$B is returned to the memory controller, indicating that the near memory includes cache line CL$B and another cache line CL$A is all zeros. In addition, the data XYZ is returned to the home agent with an included bit (INCL) set to a logic value of "1", indicating that the near memory includes cache line CL$B. Also, another bit (OTHER) is set to zero, indicating that another cache line CL$A is all zeros.
[0044] Figure 6 6 is a diagram illustrating a transaction flow 600 associated with a transaction that occurs when a home agent initiates a write cache line CL$B operation. Transaction flow 600 is Figure 5, which is a continuation of the transaction shown in . In this example, data XYZ' is to be written. The state of the bit (INCL) included in the system-level cache controller (SLCC) communication is a logic value "1" because cache line CL$B is guaranteed to be in the system-level cache. In addition to this, another bit (OTHER) is set to a logic "0" because another cache line CL$A is all zero. When there is a request to read cache line CL$A, this knowledge helps prevent reads from far memory because it can be synthesized by the memory controller. In this example, because the near memory contains cache line CL$B, the contents of the near memory are modified from XYZ to XYZ' for the address in the near memory corresponding to cache line CL$B without any interaction with the far memory.
[0045] Table 3, which is divided into three parts (3A, 3B, and 3C), shows an additional example of operations associated with the systems and methods described herein. Each part of Table 3 includes a subset of columns. Sometimes, the tables are collectively referred to as Table 3 or Tables 3A to 3C. The example assumes that near memory is implemented using DRAM and far memory is implemented using memory that complies with the Compute Express Link (CXL) specification. Therefore, the example refers to DDR transactions associated with DRAM and CXL transactions associated with CXL memory. Each row of Table 3 is similar to the one described earlier as Figure 3 Transaction flow 300, Figure 4 Transaction flow 400, Figure 5 Transaction flow 500, Figure 6 Transaction flow 600, and Figure 7 3. In addition, each of the operations described in Table 3 is described independent of the other operations.
[0046] Each operation is described by identifying the operation being performed, the starting state of various items, the bits (e.g., inclusive, other, and metatag portions) and data values being communicated, the near memory transaction and the far memory transaction (if any), and the ending state of various items. In this example, the starting state may include the state of the status bit (e.g., modified bit, exclusive bit, shared bit, or invalid bit) stored in the system-level cache, the status of the inclusive bit and another bit (e.g., CL$A and CL$B) for each of the cache lines. The starting state may also include the data value corresponding to each of the cache lines (e.g., near memory or far memory). The starting state may also include the value of the metatag portion, including the metatag value for the current row and another row, as described earlier with respect to Table 1 and the transaction flow. Similarly, in this example, the end state may include the state of a status bit (e.g., a modified bit, an exclusive bit, a shared bit, or an invalid bit) stored in the system-level cache after the operation has been completed, an included bit and another bit (e.g., CL$A and CL$B) for each of the cache lines. The end state may also include a data value corresponding to each of the cache lines (e.g., near memory or far memory). The end state may also include a value for a metatag portion, including metatag values for the current row and another row, as described earlier with respect to Table 1 and the transaction flow. Table 3A below lists example operations associated with the system described herein, and Table 3A also shows the values of the various bits and data values for the corresponding operations.
[0047]
[0048]
[0049] Form 3A
[0050] Table 3B below lists example operations associated with the systems described herein, and Table 3B also shows the values of various bits and data values for the corresponding operations.
[0051]
[0052]
[0053]
[0054] Form 3B
[0055] Table 3C below lists example operations associated with the systems described herein, and Table 3C also shows the values of various bits and data values for the corresponding operations.
[0056]
[0057]
[0058] Form 3C
[0059] Continuing with reference to Table 3, a read operation Rd A is described. Operation Rd A is associated with a situation where the CPU (or a core associated with the CPU) attempts to read a cache line stored at address A (CL$A). As shown in Table 3A, the MESI state for both the cache line stored at address A and the cache line at address B (CL$B) is invalid (I). The near memory has a cache line CL$A and the far memory has a cache line CL$B. Because the cache lines are stored in a direct exchange cache manner, one of CL$A and CL$B can be in the near memory at a time. The metadata portion (meta current line and meta other line) indicates that the current cache line in the near memory is CL$A and the other cache line (e.g., CL$B) includes zeros. Table 3B shows that as part of operation Rd A, the memory controller (MC) sends values for the included bit (Inc) and another bit (Other) to the system-level cache controller (SLCC). The included bit is set to 1, and in this way, the MC can make the SLCC aware that cache line CL$A is in near memory. Another bit is set to 1, and in this way, the MC can make the SLCC aware that another cache line (CL$B) is all zero. Because the near memory has a cache line (CL$A) that is read, it is a hit; therefore, as shown in Table 3B, there is a near memory transaction (e.g., a DDR transaction), but there is no far memory transaction (e.g., a CXL transaction). The value of the cache line that is read is provided to the requesting entity. As shown in Table 3C, in terms of the end state, because the read operation is related to cache line CL$A, there is no change relative to the bit value associated with cache line CL$B. The MESI (modify, exclusive, share, or invalid) state for cache line CL$A is changed from invalid (I) to one of modify (M), exclusive (E), or share (S). In addition, because cache line CL$A is now guaranteed to be in near memory, the included bits sent by the MC are maintained by the SLCC on the system level cache side. Finally, because this is a read operation, there are no changes to both the metadata information (meta current line and meta other lines) and the data values in near or far memory.
[0060] Still referring to Tables 3A to 3C, a write operation Wr A is described with reference to the second row (not counting the row header) of each table in Tables 3A to 3C. This example assumes that cache line CL$A is now dirty and needs to be written to near memory due to an eviction from the system-level cache. This is indicated by the status bit as modified (M). When performing operation Rd A, the other values are the same as before. Table 3B shows that as part of operation Wr A, the system-level cache controller (SLCC) (MC) sends values for the included bit (Inc) and another bit (Other) to the memory controller. The included bit is set to 1, indicating that cache line CL$A is guaranteed to be in near memory. This means that the MC can perform a blind write operation without having to perform a read before writing. Another bit is set to 0, and in this way, the MC knows that the other cache line (CL$B) is all zeros, and this fact is still tracked. Because the cache line (CL$A) is written to near memory, there is a near memory transaction (e.g., a DDR transaction) as shown in Table 3B, but there is no far memory transaction (e.g., a CXL transaction). As shown in Table 3C, the end state is the same as before, except that the state of cache line CL$A in the system level cache is now invalid (I).
[0061] Continuing to refer to Tables 3A to 3C, the Rd B operation in the third row is a read hit condition, and another cache line state is sent by the MC to the SLCC. The Wr B operation in the fourth row is a write operation performed while tracking the all-zero state of another cache line.
[0062] Next, the operation (Rd B) in the fifth row of Table 3 is described. In this case, cache line CL$B is in far memory and is all zeros. Therefore, the memory controller (MC) does not need to fetch the cache line from far memory; instead, the MC synthesizes zeros and provides the zeros to the requesting entity. In summary, due to the use of metadata (metadata other lines), when DDR transactions occur, CXL transactions do not occur - saving power and time.
[0063] Still referring to Tables 3A to 3C, a write operation Wr B is described with reference to the sixth row (not counting the row header) of each of Tables 3A to 3C. This example is related to the write of cache line CL$B and is used to illustrate the automatic swap operation. As shown in Table 3B, the memory controller (MC) communicates to the system level cache controller (SLCC) that the state of the bit and another bit is logic "0". In this example, there is first a read request from the memory controller to read cache line CL$B. Using this read request, the memory controller (e.g., earlier with respect to Figure 13C) examines the contents of the near memory. The metatag portion in this example is 00, indicating that the near memory includes the cache line CL$A that caused the miss. However, the metatag portion also indicates that another cache line (CL$B) in the far memory includes only zeros. In this case, instead of retrieving cache line CL$B from the far memory, the memory controller performs an automatic swap operation. The automatic swap operation includes the memory controller writing data to the near memory for cache line CL$B and changing the metatag portion from 00 to 10, indicating that cache line CL$B does not include all zeros (reflected in the end state in the sixth row of Table 3C). Because the cache line has all zeros in both the near memory locations and the far memory locations, no changes are made with respect to the metatag portion (00) associated with cache line CL$A. Advantageously, as a result, no access to the far memory is required, saving power and time.
[0064] Referring to Tables 3A to 3C, only operations starting from the seventh row (operation Rd B) to the fourteenth row (operation Wr B) are summarized. The Rd B operation in the seventh row of Table 3 is a conventional swap operation that causes the contents of the near memory and the contents of the far memory to be swapped with each other. The Wr B operation in the eighth row of Table 3 is a normal write operation. The Wr0 B operation in the ninth row of Table 3 is noteworthy because, due to the metadata (meta other lines are zeros) tracking the state of the cache line as all zeros in the far memory, the cache line CL$B is not written to the far memory. The Rd A operation in the tenth row of Table 3 is also noteworthy because, due to the metadata (meta other lines are zeros) tracking the state of the cache line as all zeros in the far memory, the cache line CL$B is not written to the far memory.
[0065] Still referring to Tables 3A to 3C, the Rd B in the eleventh row of Table 3 is a conventional read hit case. It is noteworthy that another row state (meta other row) is sent by the memory controller (MC) to the system-level cache controller (SLCC). The Wr B operation in the twelfth row of Table 3 is a write operation and includes the retention of tracking of the non-zero state of the other row via metadata (meta other row). The Rd B operation in the thirteenth row is a read operation that causes the memory controller (MC) to synthesize zeros and does not generate a far memory transaction. As part of the Wr B operation in the fourteenth row of Table 3, cache line CL$A is swapped out for cache line CL$B without requiring a read of far memory, and thus saving the power and time required for such an operation.
[0066] Continuing to refer to Tables 3A to 3C, in the fifteenth and sixteenth rows, non-temporal (NT) operations are described. This non-temporal write operation (NT Wr operation) is usually performed relative to a memory area marked as weakly ordered. This operation is different because, unlike other writes that need to be read before writing, the NT write operation is performed without a prior read operation. As an example, when the goal is to clear the memory content, the NT write operation can be used. Assuming that a virtual machine (VM) or an operating system (OS) wants to clear a certain amount of memory, it starts writing zeros to each address in the address range associated with the relevant memory by using a non-temporal write operation that does not need to be read before writing. The NTWr0 A operation in the fifteenth row of Table 3 is an example non-temporal write operation relative to cache line CL$A. In this example, it is assumed that cache line CL$B is all zero. The system-level cache controller (SLCC) indicates to the memory controller (MC) that the state of the bit is invalid and does not care about the state of the bit and another bit included. Because this is a hit, a DDR read transaction occurs, followed by a write to the near memory. CXL transactions do not occur. Tracking of the all-zeros for CL$B is maintained by the SLCC by indicating to the MC that the value of the metadata (meta other line) is logical "1". The NTWr0 B operation in row sixteen of Table 3 is an example of a non-temporal write operation relative to cache line CL$B. In this case, because cache line CL$B is in far memory (e.g., in a CXL-attached device), the write operation results in a miss. Because this is a non-temporal write operation, the included bits are not set in this case. The data requested to be written is all zeros. Although the near memory has cache line CL$A (see the start state in row sixteen of Table 3A), and the write operation is to cache line CL$B, rather than performing a write operation to far memory (and generating a CXL transaction), the metadata (meta other line) is set back to a logical value of "0". In other words, the value of cache line CL$B in far memory is independent of this non-temporal write operation.
[0067] Figure 7 An example transaction flow 700 is shown relating to a read operation and a subsequent write operation with speculative exchange. As explained earlier, during a read operation, the CPU (e.g., Figure 1 Any CPU 112, CPU 142, or CPU 172) may issue a request that is sent by a memory controller (e.g., Figure 1 148, and memory controller 178) associated with the home agent to read data. In this example, there is first a read request from the home agent to read cache line CL$B. The memory controller (e.g., earlier with respect to Figure 1 ) examines the contents of the near memory. The metatag portion in this example is "00", indicating that the near memory includes the cache line CL$A that caused the miss. In this example, the metatag portion also indicates that another cache line (CL$B) in the far memory includes only zeros. As a result, the included bit (INCL) is indicated as a logical value of "0" and another bit (OTHER) is also indicated as a logical value of "0". It is worth noting that in this case (as described earlier with respect to Figure 3 300 ), instead of retrieving cache line CL$B from far memory, the memory controller synthesizes the data portion corresponding to cache line CL$B to generate a synthesized data portion (all zeros), and sends the synthesized data portion and the metadata portion to the home agent. Advantageously, as a result, no access to far memory is required, saving both power and time.
[0068] Continue to refer Figure 7 , in this example as shown via the dashed portion of the transaction flow, after servicing the write operation for cache line CL$B, the memory controller (MC) may perform a speculative swap by writing cache line CL$A to far memory and updating the metadata portion in near memory to reflect the fact that near memory now includes the speculatively swapped cache line CL$A. After updating the metadata portion, the memory controller may write cache line CL$A to far memory. Notably, if cache line CL$A is indicated as all zeros, the memory controller does not need to perform a write operation to far memory because the metadata portion (metadata line) simply reflects that state.
[0069] Still viewing Figure 7 , the bottom portion of transaction flow 700 shows transactions associated with a write cache line CL$B operation. The benefit of the speculative swap operation described above is that if the next transaction involves a write cache line CL$B, the SLCC can use the previously set included bits and perform a write operation to the near memory for cache line CL$B without looking up the value in the near memory. In this example, data XYZ will be written. Because cache line CL$B is guaranteed to be in the system-level cache, the system-level cache controller (SLCC) communicates that the state of the included bit (INCL) is a logical value of "1". In addition to this, another bit (OTHER) is set to logic "1" because another cache line CL$A is not all zero. In this example, the contents of the near memory are modified from 00 to XYZ for the address in the near memory corresponding to cache line CL$B without any interaction with the far memory.
[0070] Figure 8 A block diagram of an example system 800 for implementing at least some methods of integrated memory pooling and direct exchange caching is shown. The system 800 may include (multiple) processors 802, (multiple) I / O components 804, memory 806, (multiple) presentation components 808, (multiple) sensors 810, (multiple) databases 812, networking interfaces 814, and (multiple) I / O ports 816, which may be interconnected via a bus 820. The (multiple) processors 802 may execute instructions stored in the memory 806. The (multiple) I / O components 804 may include components such as a keyboard, a mouse, a voice recognition processor, or a touch screen. The memory 806 may be any combination of non-volatile storage devices or volatile storage devices (e.g., flash memory, DRAM, SRAM, or other types of memory). The (multiple) presentation components 808 may include a display, a holographic device, or other presentation devices. The display may be any type of display, such as an LCD, LED, or other type of display. The sensor(s) 810 may include telemetry or other types of sensors configured to detect information and / or receive information (e.g., collected data). The sensor(s) 810 may include telemetry or other types of sensors configured to detect information and / or receive information (e.g., memory usage of various computing entities executed by various computing nodes in a data center). The sensor(s) 810 may include sensors configured to sense conditions associated with a CPU, memory or other storage component, FPGA, motherboard, baseboard management controller, or the like. The sensor(s) 810 may also include sensors configured to sense conditions associated with a rack, chassis, fan, power supply (PSU), or the like. The sensor(s) 810 may also include sensors configured to sense conditions associated with a network interface controller (NIC), a top-of-rack (TOR) switch, a middle-of-rack (MOR) switch, a router, a power distribution unit (PDU), a rack-level uninterruptible power supply (UPS) system, or the like.
[0071] Still reference Figure 8 , (multiple) databases 812 can be used to store any of the collected or recorded data and as required for the performance of the methods described herein. (Multiple) databases 812 can be implemented as a batch of distributed databases or a single database. (Multiple) network interfaces 814 may include communication interfaces, such as Ethernet, cellular radio, Bluetooth radio, UWB radio, or other types of wireless communication interfaces or wired communication interfaces. I / O ports 816 may include Ethernet ports, fiber optic ports, wireless ports, or other communication ports or diagnostic ports. Although Figure 8System 800 is shown as including a certain number of components arranged and coupled in a certain manner, but it may include fewer components or additional components arranged and coupled in a different manner. In addition, the functionality associated with system 800 may be distributed as desired.
[0072] Fig. 9 A data center 900 for implementing a system for integrated memory pooling and direct exchange caching according to an example is shown. As an example, the data center 900 may include a plurality of rack clusters including platform hardware, such as computing resources, storage resources, networking resources, or other types of resources. The computing resources may be provided by computing nodes provided via servers, which may be connected to switches to form a network. The network may enable connections between each possible combination of switches. The data center 900 may include server 1 910 and server N 930. The data center 900 may also include data center related functions 960, including deployment / monitoring 970, directory / identity services 972, load balancing 974, data center controllers 976 (e.g., software defined networking (SDN) controllers and other controllers), and routers / switches 978. Server 1 910 may include (multiple) CPUs 911, a host hypervisor 912, near memory 913, (multiple) storage interface controllers ((multiple) SICs) 914, far memory 915, network interface controllers ((multiple) NICs) 916, and storage disks 917 and 918. Far memory 915 may be implemented as pooled memory, as explained earlier. Server N 930 may include (multiple) CPUs 931, a host hypervisor 932, near memory 933, (multiple) storage interface controllers ((multiple) SICs) 934, far memory 935, (multiple) network interface controllers ((multiple) NICs) 936, and storage disks 937 and 938. Far memory 935 may be implemented as pooled memory, as explained earlier. Server 1 910 may be configured to support virtual machines, including VM1 919, VM2 920, and VMN 921. The virtual machines may also be configured to support applications such as APP1 922, APP2 923, and APPN 924. The server 930 may be configured to support virtual machines including VM1 939, VM2 940, and VMN 941. The virtual machines may also be configured to support applications such as APP1 942, APP2 943, and APPN 944.
[0073] Continue to refer Fig. 9In one example, a data center 900 may be enabled for multiple tenants using a virtual extensible local area network (VXLAN) framework. Each virtual machine (VM) may be allowed to communicate with VMs in the same VXLAN segment. Each VXLAN segment may be identified by a VXLAN network identifier (VNI). Fig. 9 Data center 900 is shown as including a certain number of components arranged and coupled in a certain manner, but the data center may include fewer components or additional components arranged and coupled in a different manner. In addition, the functions associated with data center 900 may be distributed or combined as desired.
[0074] Fig.10 A flowchart 1000 of an example method for managing a memory having near memory and far memory is shown. In one example, the steps associated with the method can be performed by the system described earlier (e.g., Figure 1 System 100 and Figure 2 Step 1010 may include receiving a request from a requester to read a data block stored in a near memory or a far memory. As an example, the read request may be a read request from a home agent, such as Figure 3 The home agent may be a last-level cache controller (e.g., Figure 1 any SLCC 116, SLCC 146, and SLCC 176) or any other controller that controls the coherency of a given cache line.
[0075] Step 1020 may include analyzing a metadata portion associated with the data block, the metadata portion including both: (1) first information regarding whether the near memory includes the data block or whether the far memory includes the data block and (2) second information regarding whether a data portion associated with the data block is all zeros. The memory controller (e.g., earlier with respect to Figure 1 Any near memory controller described herein) can analyze the metadata portion. As an example, the meta tag portion in one example can be "00", indicating that the near memory does not include the data block, but rather that the data block is stored in the far memory and is all zeros. This is similar to relative to Figure 3 3 , where the meta-tag portion indicates that another cache line (CL$B) in far memory includes only zeros.
[0076] Step 1030 may include, in response to determining that the far memory includes a data block and determining that a data portion associated with the data block is all zeros, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requester. Notably, in this case, instead of retrieving the data block (e.g., cache line CL$B) from the far memory, the memory controller may synthesize the data portion corresponding to the data block to generate a synthesized data portion (all zeros) and send the synthesized data portion to the home agent. Advantageously, as a result, no access to the far memory is required, saving both power and time. In one example, the synthesized data portion (e.g., all zeros) corresponding to the data portion (e.g., 512 bits) may be received by the near memory controller (e.g., Figure 1 Any memory controller 118, memory controller 148, and memory controller 178) is generated by taking 512 bits that are grounded and therefore effectively zero and providing those 512 bits in response to a read operation.
[0077] Fig.11 A flowchart 1100 of an example method for managing a memory having near memory and far memory is shown. In one example, the steps associated with the method can be performed by the system described earlier (e.g., Figure 1 System 100 and Figure 2 Step 1110 may include receiving a request from a requester to read a data block stored in a near memory or a far memory. As an example, the read request may be a read request from a home agent, such as Figure 7 The home agent may be a last-level cache controller (e.g., Figure 1 any SLCC 116, SLCC 146, and SLCC 176) or any other controller that controls the coherency of a given cache line.
[0078] Step 1120 may include analyzing a metadata portion associated with the data block, the metadata portion including both: (1) first information regarding whether the near memory includes the data block or whether the far memory includes the data block and (2) second information regarding whether a data portion associated with the data block is all zeros. The memory controller (e.g., earlier with respect to Figure 1 Any near memory controller described herein) can analyze the metadata portion. As an example, the meta tag portion in one example can be "00", indicating that the near memory does not include the data block, but rather that the data block is stored in the far memory and is all zeros. This is similar to relative to Figure 7, where the meta-tag portion indicates that another cache line (CL$A) in far memory includes only zeros.
[0079] Step 1130 may include, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zeros, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requester. Notably, in this case, instead of retrieving the data block (e.g., cache line CL$A) from the far memory, the memory controller may synthesize the data portion corresponding to the data block to generate a synthesized data portion (all zeros) and send the synthesized data portion and the metadata portion to the home agent. Advantageously, as a result, no access to the far memory is required, saving both power and time.
[0080] Step 1140 may include performing a speculative swap by writing to the far memory a portion of data corresponding to the data block previously stored in the near memory and updating the metadata portion stored in the near memory to reflect that the near memory includes the portion of data corresponding to the data block previously stored in the far memory. Figure 7 After servicing the write operation to cache line CL$B as described in transaction flow 700 of FIG. 7 , the memory controller (MC) may perform a speculative swap by writing the current data block in the near memory (e.g., cache line CL$A) to the far memory and updating the metadata portion in the near memory to reflect the fact that the near memory now includes the speculatively swapped data block (e.g., cache line CL$A). After updating the metadata portion, the memory controller may write the data block (e.g., cache line CL$A) to the far memory.
[0081] In summary, the present disclosure relates to a method for managing a system having near memory and far memory. The method may include receiving a request from a requestor to read a data block stored in the near memory or in the far memory. The method may also include analyzing a metadata portion associated with the data block, the metadata portion including both: (1) first information related to whether the near memory includes the data block or whether the far memory includes the data block and (2) second information related to whether a data portion associated with the data block is all zero. The method may also include, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zero, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requestor.
[0082] Each data block includes a cache line for a central processing unit (CPU) associated with the system. As part of the method, synthesizing data portions corresponding to the data blocks to generate synthesized data portions may include providing, by a memory controller associated with the near memory, a grounding bit associated with the data portion.
[0083] The method also includes a system-level cache controller associated with the system, tracking information about whether the cache line is guaranteed to be located in the near memory. The method also includes the system-level cache controller tracking information about whether the cache line located in the far memory includes all zeros. The near memory may include a first swappable range of memory addresses assignable to a process executed by the CPU, wherein the far memory may include a second swappable range of memory addresses assignable to a process executed by the CPU, and a ratio of a size of the first swappable range of memory addresses associated with the near memory and a size of the second swappable range of memory addresses associated with the far memory is fixed.
[0084] In another aspect, the present disclosure relates to a system having a near memory and a far memory. The system may include a near memory controller configured to receive a request from a requestor to read a data block stored in the near memory or in the far memory. The system may also include a near memory controller further configured to analyze a metadata portion associated with the data block, the metadata portion including: (1) first information related to whether the near memory includes the data block or whether the far memory includes the data block and (2) second information related to whether a data portion associated with the data block is all zero. The system may also include a near memory controller further configured to, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zero, synthesize the data portion corresponding to the data block to generate a synthesized data portion and send the synthesized data portion to the requestor instead of retrieving the data portion from the far memory.
[0085] The system may also include a central processing unit (CPU), and each data block may include a cache line for the CPU.The near memory controller may be configured to provide an access bit associated with the data portion as the synthesized data portion.
[0086] The system may also include a system-level cache controller configured to track information about whether a cache line is guaranteed to be located in the near memory. The system-level cache controller may also be configured to track information about whether a cache line located in the far memory includes all zeros. The near memory may include a first swappable range of memory addresses assignable to a process executed by the CPU, the far memory may include a second swappable range of memory addresses assignable to a process executed by the CPU, and a ratio of a size of the first swappable range of memory addresses associated with the near memory and a size of the second swappable range of memory addresses associated with the far memory is fixed.
[0087] In yet another aspect, the present disclosure relates to a method for managing a system having near memory and far memory. The method may include receiving a request from a requestor to read a data block stored in the near memory or in the far memory. The method may also include analyzing a metadata portion associated with the data block, the metadata portion including both: (1) first information regarding whether the near memory includes the data block or whether the far memory includes the data block and (2) second information regarding whether a data portion associated with the data block is all zeros.
[0088] The method may also include, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zeros, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requester. The method may also include performing a speculative swap by writing the data portion corresponding to the data block previously stored in the near memory to the far memory and updating the metadata portion stored in the near memory to reflect that the near memory includes the data portion corresponding to the data block previously stored in the far memory.
[0089] Each data block includes a cache line for a central processing unit (CPU) associated with the system. As part of the method, synthesizing data portions corresponding to the data blocks to generate synthesized data portions may include a near memory controller associated with the near memory providing a grounding bit associated with the data portion.
[0090] The method also includes a system-level cache controller associated with the system, tracking information regarding whether the cache line is guaranteed to be located in the near memory. The method also includes the system-level cache controller tracking information regarding whether the cache line located in the far memory includes all zeros.
[0091] The near memory may include a first swappable range of memory addresses assignable to a process executed by the CPU, wherein the far memory may include a second swappable range of memory addresses assignable to a process executed by the CPU, and a ratio of a size of the first swappable range of memory addresses associated with the near memory and a size of the second swappable range of memory addresses associated with the far memory is fixed. The near memory may be managed using a near memory controller, and the far memory may be managed using a far memory controller configured to communicate with the near memory controller.
[0092] It should be understood that the methods, modules, and components described herein are merely exemplary. Alternatively or in addition, the functions described herein may be at least partially performed by one or more hardware logic components. For example, but not limited to, the exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), etc. In an abstract but still clear sense, any arrangement of components that achieve the same function is effectively "associated" so that the desired function is achieved. Therefore, any two components that are combined to achieve a specific function herein can be considered to be "associated" with each other so that the desired function is independent of the architecture or intermediate components. Similarly, any two components so associated can also be considered to be "operably connected" or "coupled" to each other to achieve the desired function. Just because a component that can be any other implementation of a device, structure, system, or function is described as being coupled to another component in this article, it does not mean that the component must be a separate component. As an example, a component A described as being coupled to another component B may be a subcomponent of component B, component B may be a subcomponent of component A, or components A and B may be combined subcomponents of another component C.
[0093] Functions associated with some of the examples described in this disclosure may also include instructions stored in non-transient media. The term "non-transient media" used herein refers to any medium that stores data and / or instructions that cause a machine to operate in a particular manner. Exemplary non-transient media include non-volatile media and / or volatile media. Non-volatile media include, for example, hard disks, solid-state drives, disks or tapes, optical disks or tapes, flash memory, EPROM, NVRAM, PRAM, or other such media, or networked versions of such media. Volatile media include, for example, dynamic memory (such as DRAM, SRAM, cache) or other such media. Non-transient media are different from transmission media, but can be used in conjunction with transmission media. Transmission media are used to transfer data and / or instructions to a machine, or to transfer data and / or instructions from a machine. Exemplary transmission media include coaxial cables, fiber optic cables, copper wires, and wireless media (such as radio waves).
[0094] In addition, those skilled in the art will appreciate that the boundaries between the functions of the operations described above are merely illustrative. The functions of multiple operations may be combined into a single operation, and / or the functions of a single operation may be distributed in additional operations. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of the operations may be changed in various other embodiments.
[0095] Although the present disclosure provides specific examples, various modifications and changes may be made without departing from the scope of the present disclosure as set forth in the claims below. Therefore, the specification and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of the present disclosure. Any benefits, advantages, or solutions to problems described herein with respect to specific examples are not intended to be construed as key, essential, or necessary features or elements of any or all claims.
[0096] In addition, the terms "a" or "an" as used herein are defined as one or more. Moreover, the use of introductory phrases such as "at least one" and "one or more" in the claims should not be interpreted as implying that the introduction of another claim element by the indefinite article "a" or "an" limits any particular claim including such introduced claim element to an invention including only one such element, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "an" or "an". The same applies to the use of definite articles.
[0097] Unless otherwise stated, terms such as "first" and "second" are used to arbitrarily distinguish between the elements such terms describe. Therefore, the terms are not necessarily intended to indicate temporal priority or other priority of such elements.
Claims
1. A method for managing a system having near memory and far memory, the method comprising: receiving a request from a requester to read a data block stored in the near memory or the far memory; analyzing a metadata portion associated with the data block, the metadata portion comprising both: (1) first information regarding whether the near memory includes the data block or whether the far memory includes the data block and (2) second information regarding whether a data portion associated with the data block is all zeros; as well as In response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zeros, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requester.
2. The method of claim 1, wherein each data block comprises a cache line for a central processing unit (CPU) associated with the system.
3. The method of claim 1 , wherein synthesizing the data portion corresponding to the data block to generate the synthesized data portion comprises: A memory controller associated with the near memory provides access bits associated with the data portion.
4. The method according to claim 2, further comprising: A system-level cache controller associated with the system tracks information regarding whether a cache line is guaranteed to be in the near memory.
5. The method according to claim 4, further comprising: The system level cache controller tracks information regarding whether a cache line located in the far memory includes all zeros.
6. The method of claim 2 , wherein the near memory comprises a first swappable range of memory addresses assignable to a process for execution by the CPU, wherein the far memory comprises a second swappable range of memory addresses assignable to the process for execution by the CPU, and wherein a ratio of a size of the first swappable range of memory addresses associated with the near memory and a size of the second swappable range of memory addresses associated with the far memory is fixed.
7. A system having a near memory and a far memory, the system comprising: a near memory controller configured to receive a request from a requester to read a data block stored in the near memory or the far memory; The near memory controller is further configured to analyze a metadata portion associated with the data block, the metadata portion including both: (1) first information regarding whether the near memory includes the data block or whether the far memory includes the data block and (2) second information regarding whether a data portion associated with the data block is all zeros; and The near memory controller is further configured to, in response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zeros, instead of retrieving the data portion from the far memory, synthesize the data portion corresponding to the data block to generate a synthesized data portion and send the synthesized data portion to the requester.
8. The system of claim 7, wherein the system further comprises a central processing unit (CPU), and wherein each data block comprises a cache line for the CPU.
9. The system of claim 7, wherein the near memory controller is configured to provide an access bit associated with the data portion as the synthesized data portion.
10. The system of claim 8, further comprising a system-level cache controller configured to track information regarding whether a cache line is guaranteed to be in the near memory.
11. The system of claim 10, wherein the system-level cache controller is further configured to track information regarding whether a cache line located in the far memory includes all zeros.
12. The system of claim 8 , wherein the near memory comprises a first swappable range of memory addresses assignable to a process for execution by the CPU, wherein the far memory comprises a second swappable range of memory addresses assignable to the process for execution by the CPU, and wherein a ratio of a size of the first swappable range of memory addresses associated with the near memory and a size of the second swappable range of memory addresses associated with the far memory is fixed.
13. The system of claim 7, wherein the far memory is managed using a far memory controller configured to communicate with the near memory controller.
14. A method for managing a system having near memory and far memory, the method comprising: receiving a request from a requester to read a data block stored in the near memory or the far memory; analyzing a metadata portion associated with the data block, the metadata portion comprising both: (1) first information regarding whether the near memory includes the data block or whether the far memory includes the data block and (2) second information regarding whether a data portion associated with the data block is all zeros; In response to determining that the far memory includes the data block and determining that the data portion associated with the data block is all zeros, instead of retrieving the data portion from the far memory, synthesizing the data portion corresponding to the data block to generate a synthesized data portion and sending the synthesized data portion to the requestor; A speculative swap is performed by writing a portion of data corresponding to a data block previously stored in the near memory to the far memory, and updating a portion of metadata stored in the near memory to reflect that the near memory includes the portion of data corresponding to the data block previously stored in the far memory.
15. The method of claim 14, wherein each data block comprises a cache line for a central processing unit (CPU) associated with the system.
