Memory operation management in computing systems

Abstract

Techniques for memory operation management are disclosed herein. One exemplary technique includes, upon receiving a request to read data corresponding to a system memory section, retrieving from a first memory (151) data from a data portion of the first memory and metadata from a metadata portion of the first memory. The method can then include analyzing the data location information to determine whether the first memory currently contains data corresponding to the system memory section in the received request. In response to determining that the first memory currently contains data corresponding to the system memory section in the received request, transmit the retrieved data from the data portion of the first memory to a processor in response to the received request. If not, the method can include identifying a memory location in a second memory (153) that contains data corresponding to the system memory section, and retrieving the data from the identified memory location.

Description

Background Art

[0001] In computing, memory typically refers to a computing component used to store data for immediate access by a central processing unit (CPU) within a computer or other type of computing device. In addition to memory, a computer can also include one or more computer storage devices (e.g., a hard disk drive or HDD) that permanently store data on the computer. In operation, data such as application instructions can first be loaded from the computer storage device into memory. The CPU can then execute the application instructions loaded into memory to provide computing services such as word processing, online conferencing, etc.

Summary of the Invention

Problems to be Solved by the Invention

[0002] This summary is provided to introduce, in simplified form, a selection of concepts that are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0003] In computing devices, memory can include both cache memory and main memory. Cache memory can be ultrafast memory that acts as a buffer between main memory and the CPU, holding frequently used data and instructions for immediate use by the CPU. For example, some computers can include Double Data Rate (DDR) Synchronous Dynamic Random Access Memory (SDRAM) packaged with the CPU as cache memory for the CPU. Such cache memory is sometimes called "near memory" because it is close to the CPU. In addition to near memory, the CPU can also interface with main memory via Compute Express Link (CXL) or other suitable type of interface protocol. Main memory is sometimes called "far memory" because it is further from the CPU than near memory.

[0004] Using DDR SDRAM as cache memory for the CPU can have certain drawbacks. For example, near memory is typically more expensive than far memory, is not available for access by the operating system (OS) on the computing device, or is not even visible to the OS. Instead, the CPU has exclusive control over near memory. In addition, near memory devices such as DDR SDRAM can be very expensive. In some data center servers, the cost of DDR SDRAM used as near memory can amount to up to approximately 50 percent of the total cost of the data center server. Therefore, if near memory were visible to and accessible by the OS, the capital investment for data center servers, and the associated costs of providing various computing services from those servers, could be significantly reduced. [Means for solving the problem]

[0005] Some embodiments of the disclosed technology are directed toward implementing memory tiering in which near memory is used as a swap buffer for far memory, rather than as dedicated cache memory for the CPU within the computing device. Thus, the CPU can continue to cache data in near memory while near and far memory are exposed to the OS as addressable and allocable system memory. In some implementations, a hardware memory controller may be configured to control swapping operations at the cache line granularity (e.g., 64 bytes). Thus, the computing device requires no software intervention and does not cause any software impact. In other implementations, a memory controller having both hardware and software components may be used to control such swapping operations.

[0006] The ratio of storage space between near memory and far memory can be flexible. For example, the ratio between near memory and far memory can be any integer greater than or equal to 1:1. In the example provided, the range of system memory addresses may be covered by a combination of near memory and far memory in a 1:3 ratio. Thus, the range of system memory can be divided into four sections, for example, A, B, C, and D. Each section may contain a data portion (e.g., 512 bits) and a metadata portion (e.g., 128 bits). The data portion may be configured to contain data representing user data or instructions executable by the CPU in the computing device. The metadata portion may contain data representing various attributes of the data in the data portion. For example, the metadata portion may contain error checking and correction (ECC) bits or other appropriate types of information.

[0007] According to some embodiments of the disclosed technology, some bits (e.g., ECC bits) in the metadata portion in near memory can be configured to indicate (1) which section of the system memory range is currently held in near memory, and (2) the location of additional sections of the system memory range in far memory. For example, in the above example having four sections of system memory, eight bits in the metadata portion in near memory can be configured to indicate the aforementioned information. For example, the first pair of the first two bits can be configured to indicate which section is currently held in near memory, as follows: [Table 1] Therefore, the memory controller can easily determine that the near memory contains data from section A of the system memory when bits 1 and 2 contain 0 and 0, respectively.

[0008] In the example shown above, the first two bits correspond to near memory, while the additional six bits can be subdivided into three pairs, each corresponding to a location in far memory, which are mapped to the corresponding section of the system memory range. For example, the second, third, and fourth pairs could correspond to the first, second, or third location in far memory, respectively, as follows: [Table 2]

[0009] Therefore, the memory controller can easily determine where data from a section of system memory is located in far memory, even if it is not currently in near memory. For example, when the second pair (i.e., bits 3 and 4) contains (0,0), the memory controller may be configured to determine that the data corresponding to section A of system memory is at a first location in far memory. The above example uses 8 bits in the metadata portion to encode the location of individual sections within a range of system memory, but other implementations may use any other appropriate number of bits in the metadata portion to encode the same information. For example, in the illustrated example above, which has four sections, 5, 6, or 7 bits may be used to encode the section location information.

[0010] Using data from the metadata portion in near memory, the memory controller may be configured to manage swap operations between near memory and far memory in order to use near memory as a swap buffer. For example, during a read operation, the memory controller may be configured to read from near memory and retrieve data from both the data portion and the metadata portion of near memory. The memory controller may then be configured to use the above table to determine which section of system memory the retrieved data corresponds to and whether the determined section matches the target section to be read. For example, if the target section is section A and the first two bits from the metadata portion contain (0,0), the memory controller may be configured to determine that the retrieved data is from section A. Thus, the memory controller can transfer the data retrieved from section A to the requesting entity, such as an application or OS running on the computing device.

[0011] On the other hand, if the first two bits from the metadata portion contain (0,1) instead of (0,0), the memory controller may be configured to determine, for example, that the retrieved data belongs to section B ("B data") rather than section A ("A data"). The memory controller can then proceed to examine additional bits in the metadata portion and determine which bit pairs contain (0,0). For example, if the second pair from the metadata portion (bits 3 and 4) contains (0,0), the memory controller may be configured to determine that the A data is located at a first location in far memory. In response, the memory controller may be configured to read the A data from the first location in far memory and provide the A data to the requesting entity. The memory controller may then be configured to write the retrieved A data to near memory and the previously retrieved B data to a first section in far memory. The memory controller may also be configured to modify bits in the metadata portion in near memory to reflect the swapping of data between section A and section B in near memory.

[0012] During a write operation, the memory controller may be configured to first read data from the metadata portion in near memory. The memory controller may then be configured to determine which section of system memory data is currently held in near memory and to determine if the determined section matches the target section to be written to. For example, if the target section for the write operation is section A, and the first two bits from the metadata portion contain (0,0), the memory controller may be configured to determine that data A is currently in near memory. In response, the memory controller may be configured to overwrite the data in the data portion in near memory and report the completion of the write operation.

[0013] On the other hand, if the first two bits from the metadata portion contain (0,1), the memory controller may be configured to determine that data B is currently in near memory. In response, the memory controller may refrain from writing to near memory and instead continue examining the remaining bits of the metadata portion to determine which bit pairs contain (0,0). For example, if the second pair from the metadata portion (bits 3 and 4) contains (0,0), the memory controller may be configured to determine that data A is currently located at a first location in far memory. In response, the memory controller may be configured to write to the first location in far memory instead of near memory. Once complete, the memory controller may be configured to report the completion of the write operation.

[0014] Some embodiments of the disclosed technology can improve the operation and performance of computing devices by allowing memory previously used as cache memory and invisible to the OS to be configured by the OS as addressable system memory. For example, instead of using near memory as dedicated cache memory for the CPU, near memory can be used as addressable system memory while continuing to provide caching functionality to the CPU through the swapping operation described above. By increasing the amount of addressable system memory, computing or other suitable types of latency in computing devices can be reduced. [Brief explanation of the drawing]

[0015] [Figure 1] This is a schematic diagram illustrating a distributed computing system that implements memory operation management according to an embodiment of the disclosed technology.

[0016] [Figure 2]Schematic diagram showing some hardware / software components of the distributed computing system of FIG. 1 according to an embodiment of the disclosed technology.

[0017] [Figure 3] A and B are schematic diagrams showing an example of the hierarchical structure of the system memory according to an embodiment of the disclosed technology.

[0018] [Figure 4] A and B are schematic timing diagrams showing an exemplary read operation using near memory as a swap buffer according to an embodiment of the disclosed technology.

[0019] [Figure 5] A and B are schematic timing diagrams showing an exemplary write operation using near memory as a swap buffer according to an embodiment of the disclosed technology.

[0020] [Figure 6] It is a computing device suitable for certain components of the distributed computing system of FIG. 1.

Mode for Carrying Out the Invention

[0021] Certain embodiments of systems, devices, components, modules, routines, data structures, and processes for memory operation management are described below. In the following description, specific details of the components are included to provide a thorough understanding of certain embodiments of the disclosed technology. Those skilled in the art will also understand that the present technology may have additional embodiments. The present technology may also be implemented without some of the details of the embodiments described below with reference to FIGS. 1-6. For example, instead of being implemented in a data center or other suitable distributed computing system, aspects of the memory operation management techniques disclosed herein may also be implemented on a personal computer, smartphone, tablet, or other suitable type of computing device.

[0022] As used herein, the term "distributed computing system" generally refers to an interconnected computer system having a plurality of network nodes that interconnect multiple servers or hosts with each other and / or to an external network (e.g., the Internet). The term "network node" generally refers to a physical network device. Exemplary network nodes include routers, switches, hubs, bridges, load balancers, security gateways, or firewalls. A "host" generally refers to a physical computing device. In certain embodiments, a host can be configured to implement, for example, one or more virtual machines, virtual switches, or other suitable virtualization components. For example, a host can include a server having a hypervisor configured to support one or more virtual machines, virtual switches, or other suitable types of virtual components. In other embodiments, a host can be configured to directly execute a suitable application on an operating system.

[0023] Computer networks can, in certain implementations, be conceptually divided into an underlay network and an overlay network implemented on top of it. An “overlay network” generally refers to an abstract network implemented on top of and operating on top of an underlay network. An underlay network can include multiple physical network nodes interconnected with one another. An overlay network can include one or more virtual networks. A “virtual network” generally refers to an abstraction of a portion of the underlay network within an overlay network. A virtual network can include one or more virtual endpoints, called “tenant sites,” used individually by users or “tenants” to access the virtual network and associated computing, storage, or other appropriate resources. A tenant site can host one or more tenant endpoints (“TEPs”), such as virtual machines. A virtual network can interconnect multiple TEPs on different hosts. Virtual network nodes within an overlay network can be connected to one another by virtual links, each corresponding to one or more network routes along one or more physical network nodes in the underlay network. In other implementations, the computer network may include only an underlay network.

[0024] Furthermore, as used herein, the term “near memory” generally refers to memory that is physically closer to the processor (e.g., CPU) than other “far memory” located at a certain distance from the processor. For example, near memory may include one or more DDR SDRAM dies incorporated into an integrated circuit (IC) component package with one or more CPU dies via interposers and / or through-silicon vias. In contrast, far memory may include additional memory on accelerators, memory buffers, or smart I / O devices that the CPU may interface with via CXL or other suitable types of protocols. For example, in a data center, multiple memory devices on multiple servers / server blades may be pooled so that they can be assigned to a single CPU on one of those servers / server blades. The CPU can access the assigned far memory via a computer network within the data center.

[0025] Figure 1 is a schematic diagram showing a distributed computing system 100 implementing memory operation management according to an embodiment of the disclosed technology. As shown in Figure 1, the distributed computing system 100 may include an underlay network 108 interconnecting a plurality of hosts 106, a plurality of client devices 102 associated with corresponding users 101, and a platform controller 125 that is operationally coupled to one another. The platform controller 125 may be a cluster controller, a fabric controller, a database controller, and / or other suitable type of controller configured to monitor and manage the resources and operations of the servers 106 and / or other components within the distributed computing system 100. While the components of the distributed computing system 100 are shown in Figure 1, in other embodiments the distributed computing system 100 may also include additional and / or different components or configurations. For example, in certain embodiments the distributed computing system 100 may also include a network storage device, additional hosts, and / or other suitable components (not shown) in other suitable configurations.

[0026] As shown in Figure 1, the underlay network 108 may include one or more network nodes 112 that interconnect multiple hosts 106 and user 101's client devices 102. In certain embodiments, the hosts 106 may be organized into racks, action zones, groups, sets, or other preferred divisions. For example, in the illustrated embodiment, the hosts 106 are grouped into three host sets, individually identified as the first, second, and third host sets 107a-107c. Each of the host sets 107a-107c is operationally coupled to a corresponding network node 112a-112c, commonly referred to as a “top of rack” network node or “TOR”. The TORs 112a-112c are then operationally coupled to additional network nodes 112 to form a computer network in a hierarchical, flat, mesh, or other suitable type of topology. The underlay network can allow communication between host 106, platform controller 125, and user 101. In other embodiments, multiple host sets 107a-107c can share a single network node 112 or have other suitable configurations.

[0027] Host 106 can be individually configured to provide computing, storage, and / or other appropriate cloud or other appropriate type of computing services to user 101. For example, as will be described in more detail below with reference to Figure 2, one of the hosts 106 can start and maintain one or more virtual machines 144 (shown in Figure 2) or containers (not shown) upon request from user 101. User 101 can then use the provided virtual machines 144 or containers to perform database, computing, communication, and / or other appropriate tasks. In certain embodiments, one of the hosts 106 can provide virtual machines 144 to multiple users 101. For example, host 106a can host three virtual machines 144, each corresponding to users 101a to 101c. In other embodiments, multiple hosts 106 can host virtual machines 144 for users 101a to 101c.

[0028] Each client device 102 may include a computing device that facilitates users 101 accessing computing services provided by host 106 via the underlay network 108. In the illustrated embodiment, each client device 102 includes a desktop computer. In other embodiments, client devices 102 may also include a laptop computer, tablet computer, smartphone, or other suitable computing device. Three users 101 are shown in Figure 1 for illustrative purposes, but in other embodiments, the distributed computing system 100 may facilitate access for any suitable number of users 101 to a cloud or other suitable type of computing service provided by host 106 within the distributed computing system 100.

[0029] Figure 2 is a schematic diagram showing certain hardware / software components of a distributed computing system 100 according to an embodiment of the disclosed technology. Figure 2 shows an overlay network 108' that can be implemented on top of the underlay network 108 of Figure 1. A specific configuration of the overlay network 108' is shown in Figure 2, but in other embodiments, the overlay network 108' may be configured in other suitable ways. For clarity, only certain components of the underlay network 108 of Figure 1 are shown in Figure 2.

[0030] In Figure 2 and other figures herein, individual software components, objects, classes, modules, and routines may be computer programs, procedures, or processes written as source code in C, C++, C#, Java®, and / or other suitable programming languages. A component may include, but is not limited to, one or more modules, objects, classes, routines, properties, processes, threads, executable files, libraries, or other components. A component may be in source or binary form. A component may include aspects of pre-compiled source code (e.g., classes, properties, procedures, routines), compiled binary units (e.g., libraries, executable files), or artifacts instantiated and used at runtime (e.g., objects, processes, threads).

[0031] Components within a system can take different forms within the system. For example, a system comprising a first component, a second component, and a third component can, but is not limited to, a system having a first component as a property in source code, a second component as a compiled library of binaries, and a third component as a thread generated at runtime. Computer programs, procedures, or processes may be compiled into objects, intermediate code, or machine code and presented for execution by one or more processors of personal computers, network servers, laptop computers, smartphones, and / or other suitable computing devices.

[0032] Similarly, a component may include hardware circuitry. Those skilled in the art will recognize that hardware can be considered fixed software, and software can be considered fluid hardware. As merely one example, software instructions within a component may be embedded in a programmable logic array circuit or designed as hardware circuitry with a suitable integrated circuit. Similarly, hardware may be emulated by software. Source code, intermediate code, and / or object code, as well as various implementations of associated data, may be stored in computer memory, including read-only memory, random-access memory, magnetic disk storage media, optical storage media, flash memory devices, and / or other suitable computer-readable storage media, excluding propagating signals.

[0033] As shown in Figure 2, source host 106a and destination hosts 106b and 106b' (only destination host 106b is shown with detailed components) may each include a processor 132, memory 134, network interface card 136, and packet processor 138 that are operationally coupled to each other. In other embodiments, host 106 may also include an input / output device configured to receive inputs from an operator and / or automation software controller (not shown) or other preferred type of hardware component and provide outputs to them.

[0034] The processor 132 may include a microprocessor, a cache, and / or other suitable logical devices. The memory 134 may include volatile and / or non-volatile media (e.g., ROM, RAM, magnetic disk storage media, optical storage media, flash memory devices, and / or other suitable storage media), and / or other types of computer-readable storage media configured to store data received from the processor 132 and instructions for the processor 510 (e.g., instructions for performing the methods described below with reference to Figures 5A-D). Although Figure 2 shows only one processor 132 and one memory 134 in each host 106 for illustrative purposes, in other embodiments, each host 106 may include two, six, eight, or any other suitable number of processors 132 and / or memory 134.

[0035] Source host 106a and destination host 106b may each contain instructions in memory 134 that can be executed by processor 132, which cause the individual processors 132 to provide a hypervisor 140 (identified individually as first and second hypervisors 140a and 140b) and an operating system 141 (identified individually as first and second operating systems 141a and 141b). Although hypervisor 140 and operating system 141 are shown as separate components, in other embodiments, hypervisor 140 may operate on an operating system 141 running on host 106 or on a firmware component of host 106.

[0036] The hypervisor 140 can be individually configured to create, monitor, terminate, and / or otherwise manage one or more virtual machines 144 organized within a tenant site 142. For example, as shown in Figure 2, source host 106a may provide a first hypervisor 140a that manages the first and second tenant sites 142a and 142b, respectively. Destination host 106b may provide a second hypervisor 140b that manages the first and second tenant sites 142a' and 142b', respectively. The hypervisor 140 is shown individually in Figure 2 as a software component. However, in other embodiments, the hypervisor 140 may be a firmware and / or hardware component. Each tenant site 142 may contain multiple virtual machines 144 for a particular tenant (not shown). For example, source host 106a and destination host 106b can host both tenant sites 142a and 142a' for the first tenant 101a (Figure 1). Source host 106a and destination host 106b can host both tenant sites 142b and 142b' for the second tenant 101b (Figure 1). Each virtual machine 144 can run its corresponding operating system, middleware, and / or applications.

[0037] As shown in Figure 2, the distributed computing system 100 may also include an overlay network 108' having one or more virtual networks 146 that interconnect tenant sites 142a and 142b across multiple hosts 106. For example, a first virtual network 142a interconnects a first tenant site 142a and 142a' on source host 106a and destination host 106b. A second virtual network 146b interconnects a second tenant site 142b and 142b' on source host 106a and destination host 106b. While a single virtual network 146 is shown corresponding to one tenant site 142, in other embodiments, multiple virtual networks 146 (not shown) may be configured to correspond to a single tenant site 146.

[0038] A virtual machine 144 can be configured to run one or more applications 147 to provide a suitable cloud or other suitable type of computing service to a user 101 (Figure 1). For example, a source host 106a may run an application 147 configured to provide a computing service that monitors online trading and delivers price data to multiple users 101 who are subscribed to the computing service. Virtual machines 144 on a virtual network 146 can also communicate with each other via an underlay network 108 (Figure 1), even if the virtual machines 144 are located on different hosts 106.

[0039] Communication within each virtual network 146 may be isolated from other virtual networks 146. In certain embodiments, communication may be allowed to traverse from one virtual network 146 to another, either through a security gateway or in a otherwise controlled manner. A virtual network address may correspond to one of the virtual machines 144 within a particular virtual network 146. Thus, different virtual networks 146 may use the same one or more virtual network addresses. Exemplary virtual network addresses may include IP addresses, MAC addresses, and / or other appropriate addresses. To facilitate communication between virtual machines 144, a virtual switch (not shown) may be configured to switch or filter packets directed to different virtual machines 144 via a network interface card 136 and facilitated by a packet processor 138.

[0040] As shown in Figure 2, to facilitate communication with each other or with external devices, individual hosts 106 may also include a network interface card ("NIC") 136 for interfacing with a computer network (e.g., the underlay network 108 in Figure 1). The NIC 136 may include a network adapter, LAN adapter, physical network interface, or other appropriate hardware circuitry and / or firmware to enable communication between hosts 106 by sending / receiving data (e.g., as packets) over a network medium (e.g., optical fiber) according to Ethernet®, Fibre Channel, Wi-Fi, or other appropriate physical and / or data link layer standards. In operation, the NIC 136 may facilitate communication to and from appropriate software components running on the hosts 106. Exemplary software components may include a virtual switch 141, a virtual machine 144, an application 147 running on the virtual machine 144, a hypervisor 140, or other appropriate types of components.

[0041] In certain implementations, the packet processor 138 may be interconnected to and / or integrated with the NIC 136 to facilitate network traffic behavior for performing communication security, network virtualization, network address translation, maintaining / restricting communication flow state, or other appropriate functions. In certain implementations, the packet processor 138 may include a field-programmable gate array ("FPGA") integrated with the NIC 136.

[0042] An FPGA can include an array of logic circuits and a hierarchy of reconfigurable interconnects that, after manufacturing, allow the logic circuits to be “wired together” like logic gates by the user. Thus, user 101 can configure logic blocks within the FPGA to perform complex combinational functions, or simply configure simple logic operations to synthesize equivalent functions that can be performed in hardware at much faster speed than software. In the illustrated embodiment, the packet processor 138 has one interface that is communicatively coupled to the NIC 136 and another interface that is coupled to a network switch (e.g., a top-of-rack or “TOR” switch). In other embodiments, the packet processor 138 can also include an application-specific integrated circuit (“ASIC”), a microprocessor, or other suitable hardware circuitry. In any of the embodiments described above, the packet processor 138 can be programmed by processor 132 (or appropriate software components associated therewith) to route packets within the packet processor 138 to achieve various aspects of time-dependent data delivery, as will be described in more detail below with reference to Figures 3A to 5.

[0043] During operation, processor 132 and / or user 101 (Figure 1) can configure logic circuits within packet processor 138 to perform complex combinational functions or simple logical operations and synthesize equivalent functions that can be executed in hardware much faster than software. For example, packet processor 138 may be configured to process inbound / outbound packets for individual flows according to configured policies or rules contained in a flow table such as MAT. The flow table may include data representing processing actions corresponding to each flow to enable a private virtual network with customer-supplied address space, scalable load balancers, security groups and access control lists ("ACLs"), virtual routing tables, bandwidth metering, quality of service ("QoS"), etc.

[0044] Therefore, once the packet processor 138 identifies inbound / outbound packets as belonging to a particular flow, it can apply one or more corresponding policies in the flow table before forwarding the processed packets to the NIC 136 or TOR 112. For example, as shown in Figure 2, application 147, virtual machine 144, and / or other suitable software components on source host 106a may generate outbound packets destined for other applications 147 on destination hosts 106b and 106b'. The NIC 136 on source host 106a can forward the generated packets to the packet processor 138 for processing according to certain policies in the flow table. Once processed, the packet processor 138 can forward the outbound packet to the first TOR 112a, which then forwards the packet to the second TOR 110b via the overlay / underlay networks 108 and 108'.

[0045] Next, the second TOR 112b can forward the packet to the packet processor 138 on destination hosts 106b and 106b' so that it is processed according to other policies in different flow tables on destination hosts 106b and 106b'. If the packet processor 138 cannot identify which flow the packet belongs to, it can forward the packet to processor 132 via NIC 136 for exception handling. In another example, if the first TOR 112a receives an inbound packet from destination host 106b, for example via the second TOR 112b, the first TOR 112a can forward the packet to the packet processor 138 so that it is processed according to the policy associated with the packet's flow. The packet processor 138 can then forward the processed packet to NIC 136 so that it is forwarded, for example, to application 147 or virtual machine 144.

[0046] In certain embodiments, memory 134 may include both cache memory and main memory (not shown). Cache memory may be very fast memory that acts as a buffer between main memory and processor 132 to hold frequently used data and instructions so that processor 132 can readily access them. For example, certain computers may include double data rate (DDR) synchronous dynamic random access memory (SDRAM) packaged with processor 132 as cache memory for processor 132. Such cache memory is sometimes called “near memory” because it is close to processor 132. In addition to near memory, processor 132 may also interface with main memory via Compute Express Link (CXL) or other suitable type of interface protocol. Main memory is sometimes called “far memory” because it is further from processor 132 than near memory.

[0047] Using DDR SDRAM as cache memory for processor 132 may have certain drawbacks. For example, near memory is typically more expensive than far memory, is not available for access by the operating system (OS) on the computing device, or is not even visible to the operating system (OS). Instead, processor 132 has exclusive control over near memory. In addition, near memory devices such as DDR SDRAM can be very expensive. In some data center servers, the cost of DDR SDRAM as near memory can amount to up to approximately 50 percent of the total server cost. Therefore, if near memory is visible to the operating system 141 and thus accessible, the capital investment for the server and the associated costs of providing various computing services from host 106 can be significantly reduced.

[0048] Some embodiments of the disclosed technology are directed toward implementing memory tiering in which near memory is used as a swap buffer for far memory rather than as dedicated cache memory for processor 132. Thus, processor 132 can continue to cache data in near memory while near and far memory are exposed to the operating system 141 as addressable and allocable system memory. In some implementations, a hardware memory controller may be configured to control swapping operations at cache line granularity (e.g., 64 bytes). Thus, host 106 experiences no software intervention or impact. In other implementations, a memory controller having both hardware and software components may be used to control such swapping operations.

[0049] The ratio of storage space between near memory and far memory can be flexible. For example, the ratio between near memory and far memory can be any integer greater than or equal to 1:1. In the illustrative example shown in Figure 3A, the range of system memory address 150 is covered by a combination of near memory 151 and far memory 153 in a 1:3 ratio. Thus, the range of system memory 150 can be divided into four sections 152, for example, A, B, C, and D (identified as 152A to 152D in Figure 3A, respectively). Each section can include a data section 156 (for example, 512 bits) and a metadata section 154 (for example, 128 bits). The data section 156 may be configured to include data representing user data or instructions executed on host 106. The metadata section 154 can include data representing various attributes of the data in the data section 156. For example, the metadata section 154 can include error checking and correction bits or other appropriate types of information.

[0050] According to some embodiments of the disclosed technology, some bits in the metadata portion 154 in near memory 151 can be configured to indicate (1) which section of the system memory range is currently held by near memory, and (2) the location of additional sections of the system memory range in far memory. In the above example having four sections of system memory 150, eight bits in the metadata portion 154 in near memory 151 can be configured to indicate the aforementioned information. For example, the first pair of the first two bits can be configured to indicate which section 152 is currently held in near memory 151, as follows: [Table 3] Therefore, the memory controller 135 can easily determine that the near memory 151 contains data from section A of the system memory when bit 1 and bit 2 contain 0 and 0, respectively, as shown in Figure 3A.

[0051] The first two bits correspond to near memory 151, while the additional six bits can be subdivided into three pairs, each corresponding to a location in far memory 153, as shown in Figure 3B. For example, the second, third, and fourth pairs can correspond to the first, second, or third location in far memory 153, respectively. [Table 4]

[0052] Therefore, the memory controller 135 can easily determine where data from a particular section of system memory 150 is located in far memory 153, even if it is not currently in near memory 151. For example, when the second pair (i.e., bits 3 and 4) includes (1,1), the memory controller 135 may be configured to determine that the data corresponding to section D of system memory 150 is located at a first location 158A in far memory 153. When the third pair (i.e., bits 5 and 6) includes (1,0), the memory controller 135 may be configured to determine that the data corresponding to section C of system memory 150 is located at a second location 158B in far memory 153. When the fourth pair (i.e., bits 7 and 8) includes (0,1), the memory controller 135 may be configured to determine that the data corresponding to section B of system memory 150 is located at a third location 158C in far memory 153. This is shown in Figures 3A and 3B.

[0053] Using data from the metadata portion 154 in near memory 151, the memory controller 135 may be configured to use near memory 151 as a swap buffer to manage swap operations between near memory 151 and far memory 153. For example, as shown in Figure 4A, during a read operation, the CPU may command the memory controller 135 to read data corresponding to section A when such data does not currently exist in the CPU's final-level cache. In response, the memory controller 135 may be configured to read from near memory, retrieving data from both the data portion and the metadata portion of near memory. The memory controller 135 may then be configured to use the above table to determine which section of system memory the retrieved data corresponds to, and to determine whether the determined section matches the target section to be read. For example, as shown in Figure 4A, when the target section is section A and the first two bits from the metadata portion include (0,0), the memory controller 135 may be configured to determine that the retrieved data is from section A (i.e., "A data"). Thus, the memory controller 135 can transfer the A data retrieved from section A to a requesting entity, such as an application running on the CPU on the computing device.

[0054] On the other hand, as shown in Figure 4B, when the first two bits from the metadata portion contain (0,1) instead of (0,0), the memory controller 135 may be configured to determine, for example, that the retrieved data belongs to section B (referred to as "B data") rather than section A data. The memory controller 135 can then proceed to examine additional bits in the metadata portion to determine which bit pairs contain (0,0). For example, when a second pair (bits 3 and 4) from the metadata portion contains (0,0), the memory controller 135 may be configured to determine that the A data is located at a first location in far memory. In response, the memory controller 135 may be configured to read the A data from the first location in far memory and provide the A data to the requesting entity. The memory controller 135 may then be configured to write the retrieved A data to near memory and the previously retrieved B data to a first section in far memory. The memory controller 135 can also be configured to modify bits in the metadata portion in near memory to reflect the swapping between section A and section B.

[0055] During a write operation, as shown in Figure 5A, the memory controller 135 may be configured to first read data from the metadata portion in near memory. The memory controller 135 may then be configured to determine which section of system memory data is currently held in near memory and to determine whether the determined section matches the target section to be written to. For example, if the target section is section A and the first two bits from the metadata portion contain (0,0), the memory controller 135 may be configured to determine that the A data is currently in near memory. Thus, the memory controller 135 may be configured to overwrite the data in the data portion in near memory and report the completion of the write operation.

[0056] On the other hand, if the first two bits from the metadata portion contain (0,1), the memory controller 135 may be configured to determine that data B is currently in near memory. In response, the memory controller 135 may refrain from writing to near memory and instead continue to examine the remaining bits of the metadata portion to determine which bit pairs contain (0,0). For example, if the second pair (bits 3 and 4) from the metadata portion contains (0,0), the memory controller 135 may be configured to determine that data A is located at a first location in far memory. In response, the memory controller 135 may write to the first location in far memory instead of near memory and report the completion of the write operation.

[0057] Some embodiments of the disclosed technology can improve the operation and performance of computing devices by allowing memory previously used as cache memory and invisible to the OS to be configured by the OS as addressable system memory. For example, instead of using near memory as dedicated cache memory for the CPU, near memory can be used as addressable system memory while continuing to provide caching functionality to the CPU through the swapping operation described above. By increasing the amount of addressable system memory, computing or other suitable types of latency in computing devices can be reduced.

[0058] Figure 6 shows a computing device 300 suitable for certain components of the distributed computing system 100 in Figure 1. For example, the computing device 300 could be suitable for the host 106, client device 102, or platform controller 125 in Figure 1. In a very basic configuration 302, the computing device 300 may include one or more processors 304 and system memory 306. A memory bus 308 may be used for communication between the processors 304 and the system memory 306.

[0059] Depending on the desired configuration, the processor 304 may be any type, including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. The processor 304 may include another level of caching, such as a level 1 cache 310 and a level 2 cache 312, a processor core 314, and registers 316. The exemplary processor core 314 may include an arithmetic logic unit (ALU), a floating-point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The exemplary memory controller 318 may also be used with the processor 304, or in some implementations, the memory controller 318 may be an internal part of the processor 304.

[0060] Depending on the desired configuration, the system memory 306 may be of any type, including, but not limited to, volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory), or any combination thereof. The system memory 306 may include the operating system 320, one or more applications 322, and program data 324. As shown in Figure 11, the operating system 320 may include a hypervisor 140 for managing one or more virtual machines 144. This described basic configuration 302 is shown in Figure 8 by its components within the inner dashed lines.

[0061] The computing device 300 may have additional features or functions and additional interfaces to facilitate communication between the basic configuration 302 and any other devices and interfaces. For example, a bus / interface controller 330 can be used to facilitate communication between the basic configuration 302 and one or more data storage devices 332 via a storage interface bus 334. The data storage devices 332 may be removable storage devices 336, non-removable storage devices 338, or a combination thereof. Examples of removable and non-removable storage devices include, to name a few, magnetic disk devices such as flexible disk drives and hard disk drives (HDDs), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid-state drives (SSDs), and tape drives. Exemplary computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any way or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. The terms “computer-readable storage medium” or “computer-readable storage device” exclude propagation signals and communication media.

[0062] System memory 306, removable storage device 336, and non-removable storage device 338 are examples of computer-readable storage media. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be used to store desired information and can be accessed by computing device 300. Any such computer-readable storage media may be part of computing device 300. The term “computer-readable storage media” excludes propagation signal and communication media.

[0063] The computing device 300 may also include an interface bus 340 to facilitate communication from various interface devices (e.g., output device 342, peripheral interface 344, and communication device 346) to the basic configuration 302 via a bus / interface controller 330. An exemplary output device 342 includes a graphics processing unit 348 and an audio processing unit 350 which may be configured to communicate with various external devices such as displays or speakers via one or more A / V ports 352. An exemplary peripheral interface 344 includes a serial interface controller 354 or a parallel interface controller 356 which may be configured to communicate with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices, etc.) or other peripheral devices (e.g., printers, scanners, etc.) via one or more I / O ports 358. An exemplary communication device 346 includes a network controller 360 which may be configured to facilitate communication with one or more other computing devices 362 over a network communication link via one or more communication ports 364.

[0064] A network communication link can be an example of a communication medium. A communication medium can typically be embodied by computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery medium. A “modulated data signal” can be a signal that has been set or modified in such a way that one or more of its properties encode information within the signal. By example, but not limited to, a communication medium can include wired media such as wired networks or direct wired connections, as well as wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR), and other wireless media. As used herein, the term computer-readable medium can include both storage media and communication media.

[0065] Computing device 300 may be implemented as part of a small form factor portable (or mobile) electronic device, such as a mobile phone, personal digital assistant (PDA), personal media player device, wireless webwatch device, personal headset device, application-specific device, or hybrid device containing any of the above functions. Computing device 300 may also be implemented as a personal computer, including both laptop and non-laptop computer configurations.

[0066] From the foregoing, it will be understood that while specific embodiments of this disclosure are described herein for illustrative purposes, various modifications can be made without departing from this disclosure. Furthermore, many elements of one embodiment can be combined with other embodiments, either in addition to or in place of elements of other embodiments. Thus, the Art is not limited except as provided for by the appended claims.

Claims

1. A method for managing memory in a computing device having a processor, a first memory adjacent to the processor and configured as a cache for the processor, a second memory separate from the processor and interfaced with the processor, and a memory controller configured to manage the operation of the first and second memories, the method being: The memory controller receives a request from the processor to read data corresponding to the system memory section from the processor's cache; In response to receiving the aforementioned read request, the memory controller performs the following actions: A step of retrieving data from the data portion and metadata from the metadata portion of the first memory from the first memory, wherein the metadata from the metadata portion encodes data location information of a plurality of system memory sections in the first memory and the second memory; The steps include: analyzing the data location information in the metadata extracted from the metadata portion of the first memory to determine whether the first memory currently contains the data corresponding to the system memory section in the received request; The steps include determining that the first memory currently contains data corresponding to the system memory section in the received request, and in response to the received request, transmitting the extracted data from the data portion of the first memory to the processor, method.

2. This is performed in response to determining that the first memory does not currently contain the data corresponding to the system memory section in the received request. The steps include: further analyzing the data location information in the retrieved metadata in order to identify the memory location in the second memory containing the data corresponding to the system memory section in the received request; The further step of responding to the received request by retrieving the data from the identified memory location in the second memory and providing the retrieved data to the processor, The method according to claim 1.

3. This is performed in response to determining that the first memory does not currently contain the data corresponding to the system memory section in the received request. The steps include: further analyzing the data location information in the retrieved metadata in order to identify the memory location in the second memory containing the data corresponding to the system memory section in the received request; The steps include: in response to the received request, retrieving the data from the identified memory location in the second memory and providing the retrieved data to the processor; The further step includes writing the retrieved data from the identified memory location in the second memory to the first memory, The method according to claim 1.

4. This is performed in response to determining that the first memory does not currently contain the data corresponding to the system memory section in the received request. The steps include: further analyzing the data location information in the retrieved metadata in order to identify the memory location in the second memory containing the data corresponding to the system memory section in the received request; The steps of: retrieving the data from the identified memory location in the second memory; The steps include writing the retrieved data from the identified memory location in the second memory to the first memory; The further step includes modifying the metadata in the metadata portion of the first memory to indicate that the data corresponding to the system memory section in the received request is currently in the first memory, The method according to claim 1.

5. This is performed in response to determining that the first memory does not currently contain the data corresponding to the system memory section in the received request. The steps include: further analyzing the data location information in the retrieved metadata in order to identify the memory location in the second memory containing the data corresponding to the system memory section in the received request; The steps of: retrieving the data from the identified memory location in the second memory; The steps include writing the retrieved data from the identified memory location in the second memory to the first memory; The further step includes writing the data retrieved from the first memory to the identified memory location in the second memory, The method according to claim 1.

6. This is performed in response to determining that the first memory does not currently contain the data corresponding to the system memory section in the received request. The steps include: further analyzing the data location information in the retrieved metadata in order to identify the memory location in the second memory containing the data corresponding to the system memory section in the received request; The steps of: retrieving the data from the identified memory location in the second memory; The steps include writing the retrieved data from the identified memory location in the second memory to the first memory; The steps include: writing the data retrieved from the first memory to the identified memory location in the second memory; Modify the metadata in the metadata portion of the first memory: The data corresponding to the system memory section in the received request is currently in the first memory. The further step includes indicating that data previously held in the first memory is now located in the identified memory location in the second memory, The method according to claim 1.

7. The metadata from the metadata portion includes one or more bits; The data location information includes one or more combinations of bits that correspond individually to one of the plurality of system memory sections; Analyzing the aforementioned data location information involves: Identify the combination of one or more bits; This includes determining whether the identified combination corresponds to the system memory section in the received request. The method according to claim 1.

8. System memory addressable by the operating system includes several system memory sections, The metadata from the metadata portion includes (1) a combination of bits indicating which system memory sections the first memory holds, and (2) one or more additional combinations of bits, each additional combination of bits indicating a location in the second memory for system memory sections not included in the first memory. The method according to claim 1.

9. A computing device, Processor and; A first memory adjacent to the processor and configured as a cache for the processor; A second memory, separate from the aforementioned processor and interfaced with the aforementioned processor; The system comprises a memory controller configured to manage the operation of the first and second memories, wherein the memory controller: When the memory controller receives a request from the processor to write a block of data corresponding to the system memory section to the processor's cache: Steps include: extracting metadata from the metadata portion of the first memory from the first memory, wherein the metadata from the metadata portion encodes data location information for a plurality of system memory sections in the first memory and the second memory; The steps include: analyzing the data location information in the metadata extracted from the metadata portion of the first memory to determine whether the first memory currently contains data corresponding to the system memory section in the received request; In response to determining that the first memory currently contains data corresponding to the system memory section in the received request, Write the block of data corresponding to the system memory section to the first memory; The processor is given the step of indicating the completion of writing the block of data. The memory controller has instructions that can be executed in order to perform the following: The aforementioned memory controller is: In response to determining that the first memory does not currently contain the data corresponding to the system memory section in the received request, The steps include: further analyzing the data location information in the retrieved metadata to identify the memory location in the second memory that currently contains the data corresponding to the system memory section in the received request; The steps include writing the block of data to the identified memory location in the second memory; The processor is given the step of indicating the completion of writing the block of data. The memory controller has additional instructions that can be executed to perform the following: Computing device.

Images