SYSTEM AND METHOD FOR HOSTING INTERLEAVING ACROSS MEMORY CHANNELS IMPLEMENTED ASSY ACROSS TWO OR MORE DIFFERENT MEMORY TYPES - Patent application

Abstract

The disclosed computing device may include at least one memory of a particular type having multiple memory channels and at least one memory of at least one other type having multiple links. The computing device may also include a remapping circuit configured to evenly interleave the multiple memory channels with the at least one memory of the at least one other type. Various other methods, systems, and computer-readable media are also disclosed.

Description

[Background technology]

[0001] Double data rate (DDR) memory is a common type of memory used in modern processors. DDR memory (e.g., DDR synchronous dynamic random access memory (SDRAM)) fetches data on both the rising and falling edges of the clock signal that regulates it, hence the name "double data rate."

[0002] Compute Express Link (CXL) is an open standard for high-speed central processing unit (CPU)-to-device and CPU-to-memory connections designed for high-performance data center computers. CXL device types include dedicated and general-purpose accelerators, as well as memory expansion boards and storage-class memory (i.e., CXL Type 3 devices). These CXL memory expander devices provide the host central processing unit (CPU) with low-latency access to local memory and / or non-volatile storage.

[0003] The accompanying drawings illustrate several exemplary embodiments and constitute a part of this specification, and together with the following description, demonstrate and explain various principles of the present disclosure. [Brief explanation of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram of an example system for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. [Figure 2] FIG. 10 is a block diagram of an additional exemplary system for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. [Figure 3] FIG. 1 is a flow diagram of an example method for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. [Figure 4] FIG. 1 is a block diagram illustrating an example representation of a system that hosts interleaving across memory channels implemented asymmetrically across two or more different memory types. [Figure 5] FIG. 1 is a block diagram illustrating an example logical representation of a system hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. DETAILED DESCRIPTION OF THE INVENTION

[0005] Throughout the drawings, like reference numerals and descriptions indicate similar, but not necessarily identical, elements. While the exemplary embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are herein described in detail. However, the exemplary embodiments described herein are not intended to be limited to the particular forms disclosed. Rather, the present disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims.

[0006] The present disclosure is generally directed to systems and methods for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. An exemplary computing device may include at least one memory of a particular type having multiple memory channels and at least one memory of at least one other type having multiple links. The exemplary computing device may also include remapping circuitry configured to uniformly interleave the multiple memory channels with at least one memory of at least one other type. In some examples, uniform interleaving may be achieved by treating a range of addresses as a number of logical destinations equal to or greater than the sum of the number of multiple memory channels and the number of multiple links. An additional example may include redirecting one or more of the logical destinations to a number of physical destinations that is less than the number of logical destinations. In this manner, a computing device with multiple types of memory hardware may be reprogrammed to meet various vendor requirements without requiring hardware modifications.

[0007] For illustrative purposes, the systems and methods disclosed herein employ an exemplary system that performs uniform interleaving for 12 DDR channels and four CXL memory expander devices limited to 2x16 links. For processors that do not support 14-channel interleaving or asymmetric memory sizes, adding this support using a complex hardware implementation may be undesirable due to the additional expense. The disclosed systems and methods avoid complex hardware solutions by using a software implementation in which the moderator block that supplies requests to the data fabric can treat the range of addresses as a 16-channel interleaving, and lookups through an address map within the moderator can perform logical-to-physical remapping. In one example, 12 DDR memory channels are uniformly interleaved with four CXL memory devices with the same capacity as the DDR memory, allowing these devices to be approximately rate-matched to the DDR memory. The resulting hardware configuration can appear to the system as a symmetric 16-channel interleaving. To limit a CXL memory expander device to 2x16 links, each x16 link can be branched into two x8 links, allowing four x8 CXL memory expander devices to be connected to 4x8 links. To the hardware, the system appears to have 12 DDR distributed home nodes and two CXL distributed home nodes, each hosting twice the memory of each DDR home node.

[0008] In one example, a computing device includes at least one memory of a particular type having multiple memory channels, at least one memory of at least one other type having multiple links, and a remapping circuit configured to uniformly interleave the multiple memory channels with the at least one memory of the at least one other type.

[0009] In another example, the computing device may be any of the exemplary computing devices described above, and the remapping circuitry is configured to treat the range of addresses as a number of logical destinations equal to or greater than the sum of the number of the plurality of memory channels and the number of the plurality of links, and to redirect one or more of the logical destinations to a number of physical destinations that is less than the number of logical destinations.

[0010] In another example, the computing device may be any of the example computing devices described above, and the remapping circuitry is configured to perform logical-to-physical remapping using one or more address maps.

[0011] In another example, the computing device may be any of the exemplary computing devices described above, wherein the at least one memory of a particular type includes a double data rate (DDR) memory and the at least one memory of at least one other type includes a plurality of Compute Express Link (CXL) memory expander devices, each having the same storage capacity as the DDR memory.

[0012] In another example, the computing device may be any of the exemplary computing devices described above, further including four CXL memory expander devices in the plurality of CXL memory expander devices and twelve DDR memory channels in the plurality of DDR memory channels.

[0013] In another example, the computing device may be any of the exemplary computing devices described above, and the four CXL memory expander devices are limited to 2x16 links.

[0014] In another example, the computing device may be any of the exemplary computing devices described above, and each of the 2x16 links is branched into 2x8 links to provide 4x8 links connected to four CXL memory expander devices.

[0015] In another example, the computing device may be any of the exemplary computing devices described above, where the moderator block of the computing device treats the range of addresses as a 16-channel interleave and performs logical-to-physical remapping using one or more address maps.

[0016] In another example, the computing device may be any of the exemplary computing devices described above, where a distributed home node interfacing to 12 DDR memory channels is configured to treat at least one of the one or more address maps as a 16-way interleave.

[0017] In another example, the computing device may be any of the exemplary computing devices described above, where a distributed home node interfacing to four CXL memory expander devices is configured to treat the range of addresses as an 8-way interleave.

[0018] In one example, a system may include at least one memory of a particular type having multiple memory channels, at least one memory of at least one other type having multiple links, at least one physical processor, and a physical memory comprising computer-executable instructions that, when executed by the physical processor, cause the at least one physical processor to uniformly interleave the multiple memory channels with the at least one memory of the at least one other type.

[0019] Another example may be any of the exemplary systems described above, where the instructions cause at least one physical processor to treat a range of addresses as a number of logical destinations equal to or greater than the sum of the number of memory channels and the number of links, and redirect one or more of the logical destinations to a number of physical destinations that is less than the number of logical destinations.

[0020] Another example may be any of the exemplary systems described above, where the remapping circuitry is configured to perform logical-to-physical remapping using one or more address maps.

[0021] Another example may be any of the exemplary systems described above, where at least one memory of a particular type includes double data rate (DDR) memory and at least one memory of at least one other type includes multiple Compute Express Link (CXL) memory expander devices, each having the same storage capacity as the DDR memory.

[0022] Another example may be any of the exemplary systems described above, further including four CXL memory expander devices in the plurality of CXL memory expander devices and twelve DDR memory channels in the plurality of DDR memory channels.

[0023] Another example may be any of the exemplary systems described above, where four CXL memory expander devices are limited to 2x16 links.

[0024] Another example may be any of the exemplary systems described above, where each of the 2x16 links is split into 2x8 links to provide 4x8 links connected to four CXL memory expander devices.

[0025] Another example may be any of the exemplary systems described above, where the moderator block treats the range of addresses as a 16-channel interleave and performs logical-to-physical remapping using one or more address maps.

[0026] In one example, a computer-implemented method may include at least one processor providing data communication with at least one memory of a particular type having a plurality of memory channels and at least one memory of at least one other type having a plurality of links, and the at least one processor uniformly interleaving the plurality of memory channels with the at least one memory of the at least one other type.

[0027] Another example may be any of the exemplary methods described above, where uniformly interleaving the plurality of memory channels with at least one memory of at least one other type includes: at least one processor treating a range of addresses as a number of logical destinations equal to or greater than the sum of the number of the plurality of memory channels and the number of the plurality of links; and at least one processor redirecting one or more of the logical destinations to a number of physical destinations that is less than the number of logical destinations.

[0028] A detailed description of an exemplary system for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types is provided below with reference to Figures 1-2. A detailed description of a corresponding computer-implemented method is also provided with reference to Figure 3. Additionally, a detailed description of an exemplary system for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types is provided with reference to Figures 4 and 5.

[0029] 1 is a block diagram of an example system 100 for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. As illustrated in this figure, the example system 100 may include one or more modules 102 for performing one or more tasks. As described in more detail below, the modules 102 may include a data communication module 104 and a remapping module 106. While illustrated as separate elements, one or more of the modules 102 in FIG. 1 may represent portions of a single module or application.

[0030] In certain embodiments, one or more of the modules 102 in Figure 1 may represent one or more software applications or programs that, when executed by a computing device, cause the computing device to perform one or more tasks. For example, as described in more detail below, one or more of the modules 102 may represent modules stored and configured to execute on one or more computing devices, such as the devices shown in Figure 2 (e.g., computing device 202 and / or server 206). Also, one or more of the modules 102 in Figure 1 may represent all or part of one or more special-purpose computers configured to perform one or more tasks.

[0031] 1 , the exemplary system 100 may also include one or more memory devices, such as memory 140. Memory 140 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, memory 140 may store, load, and / or maintain one or more of the modules 102. Examples of memory 140 include, but are not limited to, random access memory (RAM), read only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, any variation or combination of one or more of these, or any other suitable storage memory.

[0032] As illustrated in FIG. 1 , the exemplary system 100 may also include one or more physical processors, such as physical processor 130. Physical processor 130 generally represents any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, physical processor 130 may access and / or modify one or more of modules 102 stored in memory 140. Additionally or alternatively, physical processor 130 may execute one or more of modules 102 to facilitate hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. Examples of physical processor 130 include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a field-programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), one or more portions thereof, one or more variations or combinations thereof, or any other suitable physical processor.

[0033] 1, the exemplary system 100 may also include one or more instances of stored data, such as data store 120. Data store 120 generally represents any type or form of stored data. In one example, data store 120 includes a database, a spreadsheet, a table, a list, a matrix, a tree, or any other type of data structure. Examples of data store 120 include, but are not limited to, a range of addresses 122 and one or more address maps 124.

[0034] The example system 100 of FIG. 1 may be implemented in a variety of ways. For example, all or a portion of the example system 100 may represent a portion of the example system 200 of FIG. 2. As shown in FIG. 2, the system 200 may include a computing device 202 that communicates with a server 206 via a network 204. In one example, all or a portion of the functionality of the modules 102 may be performed by the computing device 202, the server 206, and / or any other suitable computing system. As described in more detail below, one or more of the modules 102 of FIG. 1, when executed by at least one processor of the computing device 202 and / or the server 206, may enable the computing device 202 and / or the server 206 to host interleaving across memory channels implemented asymmetrically across two or more different memory types.

[0035] Computing device 202 generally represents any type or form of computing device capable of reading computer-executable instructions. In some examples, a computing device includes a processor interfaced with multiple DDR memory channels and multiple CXL memory expander devices. Additional examples of computing device 202 include, but are not limited to, laptops, tablets, desktops, servers, mobile phones, personal digital assistants (PDAs), multimedia players, embedded systems, wearable devices (e.g., smart watches, smart glasses, etc.), smart vehicles, so-called Internet of Things devices (e.g., smart appliances, etc.), game consoles, any variation or combination of one or more of these, or any other suitable computing device.

[0036] Server 206 generally represents any type or form of computing device capable of reading computer-executable instructions. In some examples, the server includes a processor interfaced with multiple DDR memory channels and multiple CXL memory expander devices. Additional examples of server 206 include, but are not limited to, a storage server, a database server, an application server, and / or a web server configured to run particular software applications and / or provide various storage, database, and / or web services. Although shown as a single entity in FIG. 2, server 206 may include and / or represent multiple servers functioning and / or operating in conjunction with one another.

[0037] Network 204 generally represents any medium or architecture capable of facilitating communication or data transfer. In one example, network 204 may facilitate communication between computing device 202 and server 206. In this example, network 204 may facilitate communication or data transfer using wireless and / or wired connections. Examples of network 204 include, but are not limited to, an intranet, a wide area network (WAN), a local area network (LAN), a personal area network (PAN), the Internet, Power Line Communications (PLC), a cellular network (e.g., a Global System for Mobile Communications (GSM) network), one or more portions thereof, one or more variations or combinations thereof, or any other suitable network.

[0038] Many other devices or subsystems may be connected to system 100 of Figure 1 and / or system 200 of Figure 2. Conversely, not all of the components and devices shown in Figures 1 and 2 need be present to practice the embodiments described and / or illustrated herein. The above-referenced devices and subsystems may be interconnected in ways different from those shown in Figure 2. Additionally, systems 100 and 200 may employ any number of software, firmware, and / or hardware configurations. For example, one or more of the exemplary embodiments disclosed herein may be encoded as a computer program (also referred to as computer software, a software application, computer-readable instructions, and / or computer control logic) on a computer-readable medium.

[0039] As used herein, the term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media such as carrier waves, and non-transitory-type media such as magnetic storage media (e.g., hard disk drives, tape drives, floppy disks), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), BLU-RAY disks), electronic storage media (e.g., solid-state drives and flash media), and other distribution systems.

[0040]

[0023] Figure 3 is a flow diagram of an exemplary computer-implemented method 300 for hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. The steps illustrated in Figure 3 may be performed by any suitable computer-executable code and / or computing system, including system 100 of Figure 1, system 200 of Figure 2, and / or variations or combinations of one or more thereof. In one example, each of the steps illustrated in Figure 3 may represent an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which are provided in more detail below.

[0041] 3, one or more of the systems described herein may provide data communication at step 302. For example, data communication module 104, as part of computing device 202 of FIG. 2, may provide data communication with at least one memory of a particular type having multiple memory channels and at least one memory of at least one other type having multiple links, by at least one processor.

[0042] As used herein, the term "data communications" generally refers to the electronic transmission of coded information to, from, or between computers. Examples of data communications include, but are not limited to, simplex, duplex, and half-duplex communications.

[0043] As used herein, the term "memory" generally refers to electronic storage of instructions and / or data that a computer needs to access quickly. Examples of memory may include, but are not limited to, cache memory, main memory, and secondary memory. Different types of memory may differ in various aspects, such as the number of channels or links, different storage capacities, different rates, etc. For example, two or more different types of cache memory may be DDR memory and a CXL memory expander device.

[0044] As used herein, the term "channel" generally refers to a model for communication and synchronization between processes via message passing. Examples of channels include, but are not limited to, buffered channels, synchronous channels, and asynchronous channels. The term "memory channel" may be used interchangeably herein with the terms "memory," "DDR memory," and / or "distributed home node."

[0045] As used herein, the term "link" generally refers to any combination of hardware and / or software that provides a mechanism for data communication. Examples of links include, but are not limited to, point-to-point wired connections and / or communication paths provided by a switch fabric and / or a communications network. The term "link" may be used interchangeably herein with the terms "CXL memory expander device" and / or "distributed home node."

[0046] Various examples of the system described herein perform step 302 in various ways. In one example, data communications module 104, as part of computing device 202 of FIG. 2, can limit four CXL memory expander devices to 2×16 links each bifurcated into 2×8 links, resulting in a 4×8 link connected to four CXL memory expander devices. Alternatively or additionally, data communications module 104, as part of computing device 202 of FIG. 2, can utilize 12 DDR memory channels to provide read and write access.

[0047] One or more of the systems described herein may perform uniform interleaving at step 304. For example, remapping module 106, as part of computing device 202 of FIG. 2, may, by at least one processor, uniformly interleave multiple memory channels with at least one memory of at least one other type.

[0048] As used herein, the term "interleaving" generally refers to distributing data. Examples of interleaving include, but are not limited to, sequentially distributing fields or channels of different meanings in memory, processor registers, or file formats. For an interleave to be "uniform," two or more different memory address ranges can be mapped to logical destinations that are included in the same interleave. Thus, the number of logical destinations can be greater than or equal to the sum of the channels and links. If the number of logical destinations is greater than the sum of the channels and links, one or more of the logical destinations can be redirected to fewer physical destinations than the number of logical destinations.

[0049] Various examples of the system described herein perform step 304 in various manners. In one example, the remapping module 106, as part of the computing device 202 of FIG. 2, can treat a range of addresses as a number of logical destinations equal to or greater than the sum of the number of memory channels and the number of links, and redirect one or more of the logical destinations to a number of physical destinations that is less than the number of logical destinations. In some examples, the remapping module 106, as part of the computing device 202 of FIG. 2, can determine a “logical” destination home node instance for an address based on a selected channel interleave (e.g., a 16-channel interleave) in one or more moderators, and redirect the “logical” destination to a “physical” destination. In some embodiments, the remapping module 106, as part of the computing device 202 of FIG. 2, can use one or more address maps with a first plurality of distributed home nodes configured to treat at least one of the one or more address maps as a 16-way interleave (e.g., connected to 12 DDR memory channels). In additional or alternative examples, the remapping module 106 may use one or more address maps as part of the computing device 202 of Figure 2 with a second plurality of distributed home nodes configured to treat the range of addresses as an 8-way interleave. In some of these examples, the second plurality of distributed home nodes are connected to four CXL memory expander devices limited to 2x16 links each branched into 2x8 links to result in 4x8 links connected to four CXL memory expander devices.

[0050] As used herein, the term "range of addresses" generally refers to multiple unique identifiers used by a device or CPU for data tracking. Examples of address ranges include, but are not limited to, logical addresses and physical addresses. A physical address may be, for example, a memory address or the location of a memory cell in main memory. A logical address may be the address at which an item (e.g., a memory cell, a storage element, and / or a network host) appears to exist from the perspective of an executing application program. A logical address may differ from a physical address due to the operation of an address translation mechanism or mapping function.

[0051] FIG. 4 illustrates an exemplary system 400 hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. For example, the system 400 may include a processing module 402, a memory controller 406, an input / output (I / O) interface 408, and different types of memory controllers 409-411, all connected to a switch fabric 404. The I / O interface 408 may be one or more media (e.g., an I / O bus having data lines, address lines, and control lines) through which data is transmitted from internal logic to external sources and through which data is received from external sources. The switch fabric 404 may be a network topology in which network nodes are interconnected through one or more network switches. The memory controllers 406-410 may be digital circuits that manage the flow of data to and from main memory. The memory controller 406 may be configured as part of a cache subsystem and may include a coherency manager 424 that ensures uniformity of shared resource data stored in multiple local caches. Memory controllers 409-411 may control the flow of data to different types of memory, which may function as local caches, for example. Processing module 402 may be a physical processor including processor cores 412 and 414 connected to local caches 416 and 418, each having its own coherency manager 420 that ensures uniformity of shared resource data stored in local caches 416 and 418.

[0052] System 400 can perform interleaving in various ways. For example, coherency manager 420 can include an interleaving module 422 that converts transmission channels with memory into transmission channels without memory by permuting symbols according to a mapping. Furthermore, interleaving module 422 can uniformly interleave channels and / or links of different types of memory controllers 409-411 (e.g., which may represent a set of two or more channels for two or more memory devices). Uniform interleaving can treat a range of addresses as a number of logical destinations equal to or greater than the sum of the number of memory channels of a first type of memory controller 409 and the number of links of one or more other types of memory controllers (e.g., memory controllers 410 and / or 411).

[0053] System 400 can be configured and reconfigured in various ways by adapting one or more address maps used by interleave module 422 to reconfigure interleave module 422 as a remapping circuit that uses one or more address maps to perform logical-to-physical remapping. For example, the number of logical destinations can be equal to or greater than the sum of the number of memory channels and the number of links, and one or more of the logical destinations can be redirected to a fewer number of physical destinations than the number of logical destinations. Furthermore, one or more of memory controllers 409-411 can implement their own interleaving to limit and / or branch links, and the home node is reconfigured to host multiple memory devices. In this manner, the home node and I / O interface components (e.g., pin connectors) can be reconfigured to meet various requirements without requiring complex hardware solutions.

[0054] 5 shows an example logical representation of a system 500 hosting interleaving across memory channels implemented asymmetrically across two or more different memory types. This example performs remapping for 12 DDR memory channels 502 and four CXL memory expander devices 532, 534, 542, and 544, which normally have four x8 links but are limited to two x16 links 504 and 506. In this case, there are 16 logical destinations corresponding to the 12 DDR memory channels 502 and the four x8 links to the four CXL memory expander devices 532, 534, 542, and 544. However, there are 14 physical destinations corresponding to the 12 DDR memory channels 502 and the two x16 links 504 and 506, which are limited from the four x8 links.

[0055] As used herein, the term "CXL memory expander device" generally refers to a CXL Type 3 device that provides a host CPU with low-latency access to local memory and / or non-volatile storage. Examples of CXL memory expander devices include, but are not limited to, memory expansion boards and storage class memory.

[0056] In this example, system 500 may include one or more moderator blocks 540 that may have one or more remappers 550 that uniformly interleave memory channel 502 with one or more memories of another type (e.g., CXL memory expander devices 532, 534, 542, 544). To do so, one or more remappers 550 may treat a range of addresses as a uniform interleave 562 (e.g., a 16-channel interleave) with a number of logical destinations equal to or greater than the sum of the number of memory channels 502 (e.g., 12) and the number of x8 links (e.g., 4) to CXL memory expander devices 532, 534, 542, 544. Furthermore, one or more remappers 550 may redirect one or more of the logical destinations of the uniform interleave 562 to a number of physical destinations 564 that are fewer (e.g., 14) than the number of logical destinations of the uniform interleave 562 (e.g., 16).

[0057] In this example, the logical destinations of uniform interleave 562 may include logical destinations DDR0-DDR11 of twelve DDR home nodes 520 corresponding to the twelve DDR memory channels 502. Additionally, the logical destinations of uniform interleave 562 may include logical destinations 566-572 of four x8 links corresponding to four CXL home nodes CXL0-CXL3. Remapper 550 may redirect logical destinations 568 and 572 (e.g., corresponding to CXL home nodes CXL1 and CXL3) to physical destinations 574 and 576 (e.g., corresponding to CXL home nodes CXL0 and CXL2). Thus, although there may still be 16 entries in physical destination 564, these entries are no longer unique. This remapping may enable CXL home node CXL0 to host both CXL memory expander devices 532 and 534, and enable CXL home node CXL2 to host both CXL memory expander devices 542 and 544. As a result, both CXL memory expander devices 532 and 534 can be accessed through I / O pin connector 536 of CXL home node CXL0 without having to use I / O pin connector 538 of CXL home node CXL1. Similarly, both CXL memory expander devices 542 and 544 can be accessed through I / O pin connector 546 of CXL home node CXL2 without having to use I / O pin connector 548 of CXL home node CXL3. This reconfigurability can meet the needs of a variety of applications and devices without requiring complex hardware solutions.

[0058] Interleavers 530 and 540 in CXL home nodes CXL0 and CXL2 can limit four x8 links to two x16 links in one data communication path direction and branch two x16 links into four x8 links in the opposite data communication path direction. The two x8 links can be configured as an 8-channel interleave, and addresses from the 8-channel interleave can then be normalized (e.g., by taking three bits instead of four) before sending the associated traffic to the CXL memory controller. The CXL memory controller can utilize the additional bits to interleave between two x8 memory devices. Such an embodiment can limit four CXL memory expander devices 532, 534, 542, and 544 to two x16 links, each branched into two x8 links, resulting in four x8 links connected to the four CXL memory expander devices.

[0059] As described above, the systems and methods disclosed herein can uniformly interleave 12 DDR memory channels with four CXL Type 3 (CXL.mem) devices with the same capacity as the DDR memory, and approximately rate-match these four CXL Type 3 (CXL.mem) devices to the DDR memory. Home nodes interfacing to the DDR memory can be treated differently from home nodes interfacing to the CXL memory. The hardware can support a configuration in which four distributed CXL home nodes can appear as a logical extension of 12 distributed DDR home nodes, and this configuration can appear to the system as a symmetric channel interleave.

[0060] As used herein, the term "distributed home node" generally refers to devices that are controlled by one or more other devices (e.g., moderator blocks) and are physically separate but linked together using a network (e.g., a switch fabric). Examples of distributed home nodes include, but are not limited to, DDR memory, DDR memory channels, CXL memory expander devices, and CXL memory channels.

[0061] Furthermore, for processors capable of supporting 4x16 CXL Type 3 memory links, with the CXL links sharing pins with logic that may be used to host either socket-to-socket xGMI links or processor-to-I / O PCIe links, some applications may desire to limit the CXL Type 3 memory to two x16 links. Each x16 link can branch into two x8 links, and 4x8 CXL Type 3 devices can be connected to the 4x8 links thus created. To the hardware, this configuration may appear as a system with 12 DDR distributed home nodes and two CXL distributed home nodes, each hosting twice the size of the memory in each DDR home node.

[0062] For processors that do not support 14-channel interleaving or asymmetric memory sizes, adding this support using a complex hardware implementation may be undesirable. The disclosed system and method avoids complex hardware solutions by using a software implementation in which the moderator block that supplies requests to the data fabric can treat the address range as a 16-channel interleave, and lookups through an address map within the moderator can perform logical-to-physical remapping. For four CXL home nodes (e.g., numbered CXL0, CXL1, CXL2, and CXL3), requests targeted to CXL1 can instead go to CMP0, and requests targeted to CXL3 can instead go to CXL2. A distributed home node interfaced to 12 DDR channels can see this address map as a 16-way interleave. A distributed home node that interfaces to CXL memory (i.e., CXL0 and CXL2) can be programmed to treat the same address range as an 8-way interleave. In this way, the CXL memory can avoid objecting to receiving requests related to its corresponding requests and can properly normalize physical addresses before passing them to the CXL memory controller. The combination of 16-channel interleaving and 8-channel interleaving for the same address range can achieve the goal of hosting interleaving across asymmetrically implemented memory channels across two or more different memory types while avoiding the increased costs associated with relying on complex hardware implementations.

[0063] While the foregoing disclosure describes various embodiments using specific block diagrams, flow diagrams, and examples, each block diagram element, flow diagram step, operation, and / or component described and / or illustrated herein can be implemented individually and / or collectively using a wide variety of hardware, software, or firmware (or any combination thereof) configurations. Additionally, any disclosure of components stored within other components shall be considered exemplary in nature, as many other architectures may be implemented to achieve the same functionality.

[0064] In some examples, all or a portion of the example system 100 of Figure 1 may represent part of a cloud computing or network-based environment. A cloud computing environment may provide various services and applications over the Internet. These cloud-based services (e.g., software as a service, platform as a service, infrastructure as a service, etc.) may be accessible through a web browser or other remote interface. Various functionality described herein may be provided through a remote desktop environment or any other cloud-based computing environment.

[0065] In various embodiments, all or a portion of the example system 100 of FIG. 1 can facilitate multi-tenancy within a cloud-based computing environment. In other words, the modules described herein can configure a computing system (e.g., a server) to facilitate multi-tenancy for one or more of the functions described herein. For example, one or more of the modules described herein can program a server to allow two or more clients (e.g., customers) to share an application running on the server. A server so programmed can share applications, operating systems, processing systems, and / or storage systems among multiple customers (i.e., tenants). Additionally, one or more of the modules described herein can partition data and / or configuration information of a multi-tenant application by customer, such that one customer cannot access data and / or configuration information of another customer.

[0066] According to various embodiments, all or a portion of the example system 100 of Figure 1 may be implemented within a virtual environment. For example, the modules and / or data described herein may reside and / or execute within a virtual machine. As used herein, the term "virtual machine" generally refers to any operating system environment that is abstracted from computing hardware by a virtual machine manager (e.g., a hypervisor).

[0067] In some examples, all or a portion of the example system 100 of FIG. 1 may represent a portion of a mobile computing environment. The mobile computing environment may be implemented by a wide range of mobile computing devices, including mobile phones, tablet computers, e-readers, personal digital assistants, wearable computing devices (e.g., computing devices with head-mounted displays, smart watches, etc.), one or more variations or combinations thereof, or any other suitable mobile computing device. In some examples, the mobile computing environment may have one or more different capabilities, including, for example, dependence on battery power, presenting only one foreground application at any given time, remote management capabilities, touchscreen capabilities, location and movement data (e.g., provided by a global positioning system, gyroscope, accelerometer, etc.), a restricted platform that restricts system-level configuration changes and / or limits the ability of third-party software to inspect the behavior of other applications, controls that restrict application installation (e.g., provided only from approved application stores), etc. The various capabilities described herein may be provided to and / or interact with the mobile computing environment.

[0068] The process parameters and order of steps described and / or illustrated herein are provided by way of example only and can be changed as desired. For example, although the steps illustrated and / or described herein may be illustrated or described in a particular order, these steps do not necessarily have to be performed in the order illustrated or described. The various exemplary methods described and / or illustrated herein can also omit one or more of the steps described or illustrated herein or can include additional steps in addition to those disclosed.

[0069] Although various embodiments have been described and / or illustrated herein in the context of a fully functional computing system, one or more of these exemplary embodiments may be distributed as a program product in various forms, regardless of the particular type of computer-readable medium used to actually execute the distribution. The embodiments disclosed herein may also be implemented using modules that perform certain tasks. These modules may include scripts, batch files, or other executable files that may be stored on a computer-readable storage medium or within a computing system. In some embodiments, these modules may configure a computing system to execute one or more of the exemplary embodiments disclosed herein.

[0070] The foregoing description is provided to enable those skilled in the art to best utilize various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or to be limited to any precise form disclosed. Many changes and modifications are possible without departing from the spirit and scope of the present disclosure. The embodiments disclosed herein should be considered in all respects as illustrative and not restrictive. In determining the scope of the present disclosure, reference should be made to the appended claims and their equivalents.

[0071] Unless otherwise specified, the terms "connected to" and "coupled to" (and their derivatives) as used in this specification and claims should be interpreted as allowing both direct and indirect connections (i.e., via other elements or components). Additionally, the terms "a" or "an" as used in this specification and claims should be interpreted as meaning "at least one of." Finally, for ease of use, the terms "including" and "having" (and their derivatives) as used in this specification and claims are interchangeable with the term "comprising," and have the same meaning.

Claims

1. 1. A computing device comprising: at least one memory of a particular type having multiple memory channels; at least one memory of at least one other type having a plurality of links; a remapping circuit configured to uniformly interleave the plurality of memory channels with at least one memory of the at least one other type. Computing devices.

2. The remapping circuit treating a range of addresses as a number of logical destinations equal to or greater than the sum of the number of the plurality of memory channels and the number of the plurality of links; redirecting one or more of the logical destinations to a number of physical destinations that is less than the number of the logical destinations; configured to: The computing device of claim 1.

3. the remapping circuitry is configured to perform logical-to-physical remapping using one or more address maps; The computing device of claim 2.

4. the at least one memory of the particular type includes a double data rate (DDR) memory, and the at least one memory of the at least one other type includes a plurality of Compute Express Link (CXL) memory expander devices, each having the same storage capacity as the DDR memory; The computing device of claim 1.

5. four CXL memory expander devices among the plurality of CXL memory expander devices; and twelve DDR memory channels among the plurality of DDR memory channels. The computing device of claim 4.

6. The four CXL memory expander devices are limited to 2x16 links. The computing device of claim 5.

7. each of the 2x16 links is branched into a 2x8 link to provide a 4x8 link connected to the four CXL memory expander devices; The computing device of claim 6.

8. a moderator block of the computing device treating the address range as a 16-channel interleave and performing logical-to-physical remapping using one or more address maps; The computing device of claim 7.

9. a distributed home node interfacing to the twelve DDR memory channels is configured to treat at least one of the one or more address maps as a 16-way interleave; The computing device of claim 8.

10. a distributed home node interfacing to the four CXL memory expander devices is configured to treat the range of addresses as an 8-way interleave; The computing device of claim 9.

11. 1. A system comprising: at least one memory of a particular type having multiple memory channels; at least one memory of at least one other type having a plurality of links; at least one physical processor; a physical memory comprising computer-executable instructions that, when executed by the at least one physical processor, cause the at least one physical processor to uniformly interleave the plurality of memory channels with at least one memory of the at least one other type; system.

12. The computer-executable instructions include: treating a range of addresses as a number of logical destinations equal to or greater than the sum of the number of the plurality of memory channels and the number of the plurality of links; redirecting one or more of the logical destinations to a number of physical destinations that is less than the number of the logical destinations; causing the at least one physical processor to perform The system of claim 11.

13. the remapping circuitry is configured to perform logical-to-physical remapping using one or more address maps; The system of claim 12.

14. the at least one memory of the particular type includes a double data rate (DDR) memory, and the at least one memory of the at least one other type includes a plurality of Compute Express Link (CXL) memory expander devices, each having the same storage capacity as the DDR memory; The system of claim 11.

15. four CXL memory expander devices among the plurality of CXL memory expander devices; and twelve DDR memory channels among the plurality of DDR memory channels.

15. The system of claim 14.

16. The four CXL memory expander devices are limited to 2x16 links.

16. The system of claim 15.

17. each of the 2x16 links is branched into a 2x8 link to provide a 4x8 link connected to the four CXL memory expander devices; 17. The system of claim 16.

18. The moderator block treats the address range as a 16-channel interleave and performs logical-to-physical remapping using one or more address maps.

18. The system of claim 17.

19. 1. A computer-implemented method comprising: At least one processor providing data communication between at least one memory of a particular type having a plurality of memory channels and at least one memory of at least one other type having a plurality of links; the at least one processor uniformly interleaving the plurality of memory channels with at least one memory of the at least one other type. method.

20. Uniformly interleaving the plurality of memory channels with at least one memory of the at least one other type includes: the at least one processor treating a range of addresses as a number of logical destinations equal to or greater than the sum of the number of the plurality of memory channels and the number of the plurality of links; the at least one processor redirecting one or more of the logical destinations to a number of physical destinations that is less than the number of logical destinations.

20. The method of claim 19.

Images