Memory space allocation for non-power-of-2 memory controllers

Abstract

A system includes: a memory space accessible to read or write data; n memory controllers, N being an integer greater than 1 but not a power of 2; and one or more processors configured to assign addresses of the memory space to the M regions and map each of the N memory controllers to a subset of the M regions. Each subset of the M regions may include exactly J regions, J being less than M and being a power of 2. More than one memory controller may be mapped to each of the M regions. The same number of memory controllers may be mapped to each of the M regions.

Description

Background Art

[0001] Modern computing systems typically utilize multiple memory controllers connected to memory spaces for storage and access of data and code. The memory controllers are responsible for performing memory access operations, such as instructing read and write operations within the memory spaces.

[0002] Using multiple memory controllers to spread data across the memory space has the benefit of increasing overall access bandwidth. In this regard, many memory access requests can be sent to the memory in parallel, and these requests can be processed simultaneously by multiple memory controllers. However, since memory controllers tend to access memory locations that are close to each other in time and space, the access pattern may cause increased system latency because only one or a few memory controllers can process memory access requests. To address this issue, the memory controller can be specifically configured to evenly spread memory access requests across the memory space, thereby reducing latency and avoiding locality issues. For example, the memory controller can be programmed to interleave data across the memory space according to a dedicated pattern that evenly distributes memory access operations.

[0003] Dedicated memory access modes, such as modes for memory interleaving, are specifically designed to be implemented using only a certain number (specifically powers of 2, such as 2, 4, 8, 16, etc.) of memory controllers. However, if a different number of memory controllers, such as 10, 12, or 14 memory controllers, are used, the dedicated memory access mode for the power-of-2 memory controllers will no longer evenly spread the memory access requests across the memory space, and may even create new latency issues or locality issues. In other words, current solutions for distributing memory access requests across multiple memory controllers greatly limit the choice of memory controllers that can be included in the system, which may result in inefficient memory storage and access for the computing system. Summary of the invention

[0004] The present disclosure provides an improved memory access technique that increases access efficiency when using any number of memory controllers.

[0005] One aspect of the present disclosure relates to a system comprising: a memory space accessible to read or write data; N memory controllers, where N is an integer greater than 1 and not a power of 2; and one or more processors configured to: assign addresses of the memory space to M regions, where M is a plurality; and map each of the N memory controllers to a subset of the M regions.

[0006] In some examples, each subset of the M regions may include exactly J regions, where J is less than M and is a power of 2. In some examples, J may be a maximum value that is a power of 2 and less than M.

[0007] In some examples, one or more processors may be configured to map more than one memory controller to each of the M regions, and to map the same number of memory controllers to each of the M regions.

[0008] In some examples, N is equal to 5, M is equal to 5, and J is equal to 4. In other examples, N is equal to 6, M is equal to 6, and J is equal to 4.

[0009] In some examples, the memory space may include multiple system level cache memory blocks associated with multiple core processing units.

[0010] In some examples, one or more processors may be configured to: divide both memory controllers and system-level cache memory into multiple non-overlapping clusters; assign memory addresses to multiple clusters; and for at least one cluster, assign addresses of the cluster's memory space to M regions and map each of the N memory controllers to a subset of the M regions within the cluster.

[0011] In some examples, one or more processors may be configured to: assign memory addresses to clusters using a power-of-2 hash algorithm; and within at least one cluster, determine which system-level cache blocks of the cluster to assign memory addresses to using a non-power-of-2 hash algorithm.

[0012] In some examples, one or more processors may be configured to use address masks to control the granularity of cluster alternation of address allocations among multiple clusters.

[0013] In some examples, N memory controllers may be included in a chip.

[0014] Another aspect of the present disclosure relates to a method comprising: allocating, by one or more processors, addresses of a memory space accessible to read or write data to M regions, where M is a plurality; and mapping, by one or more processors, each of N memory controllers to a subset of the M regions, where N is an integer greater than 1 and not a power of 2.

[0015] In some examples, each subset of the M regions may include exactly J regions, where J is less than M and is a power of 2. In some examples, J may be a maximum value that is a power of 2 and less than M.

[0016] In some examples, more than one memory controller may be mapped to each of the M regions, and the same number of memory controllers may be mapped to each of the M regions.

[0017] In some examples, N is equal to 5, M is equal to 5, and J is equal to 4. In other examples, N is equal to 6, M is equal to 6, and J is equal to 4.

[0018] In some examples, allocating addresses of the memory space may include allocating system level cache memory blocks to the M regions.

[0019] In some examples, the method may also include: dividing both the memory controller and the system-level cache into multiple non-overlapping clusters by one or more processors; allocating memory addresses to the multiple clusters by one or more processors; and for at least one cluster, allocating addresses of the cluster's memory space to M regions by one or more processors and mapping each of the N memory controllers to a subset of the M regions within the cluster.

[0020] In some examples, the method may also include: using a power-of-2 hash algorithm by one or more processors to assign memory addresses to clusters; and within at least one cluster, determining by one or more processors which system-level cache blocks of the cluster to assign memory addresses to using a non-power-of-2 hash algorithm.

[0021] In some examples, the method may also include controlling, by the one or more processors, a granularity of cluster alternation of address allocations among the plurality of clusters using the address mask.

[0022] In some examples, the method may also include using, by the one or more processors, the address mask to control the granularity of address allocation within the cache of the memory space. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a block diagram of an example system according to aspects of the present disclosure.

[0024] Figure 2 is a block diagram of an example arrangement of memory according to aspects of the present disclosure.

[0025] Figure 3 is a diagram showing memory space allocation according to aspects of the present disclosure.

[0026] Figure 4 is a block diagram of another example system according to aspects of the present disclosure.

[0027] Figure 5 is a flow diagram of an example process according to aspects of the present disclosure. DETAILED DESCRIPTION

[0028] Overview

[0029] By dividing the memory addresses mapped to the memory controllers into a predetermined number of regions, improved access efficiency can be achieved when using any number of memory controllers. Each region of the memory address is then associated with a specific subset of the memory controllers, which contains a power of 2 memory controllers. The subsets of memory controllers can overlap, which means that the same memory controller can be used for multiple regions. By appropriately setting the number of memory controllers per region, it can be ensured that each memory controller appears in the same number of regions as other memory controllers and that each region is mapped to the same number of memory controllers. In other words, the memory controller continues to interleave data across the memory space based on the dedicated modes of the power of 2 memory controllers required, but now each memory controller is used in multiple of these dedicated modes, allowing each memory controller to be assigned to the same number of dedicated modes as other memory controllers.

[0030] For example, in a system with 10 memory controllers and memory addresses divided into 5 regions, each region of memory addresses may be mapped to 4 memory controllers or 8 memory controllers. In the case of 4 memory controllers per region, each memory controller will be mapped to 2 of the 5 regions. In the case of 8 memory controllers per region, each memory controller will be mapped to 4 of the 5 regions. Considerations for the preferred number of memory controllers mapped to each region may vary from computing system to computing system. In some cases, it may be preferred to select the highest power of 2 that does not exceed the total number of memory controllers.

[0031] The systems and techniques proposed in the present disclosure provide reduced latency and better distribution of data access operations when using non-power-of-two memory controllers. This results in an overall improvement in the efficiency of memory access operations in such non-power-of-two configurations, thereby creating possibilities for new combinations and arrangements of memory controllers. Increasing the available choices of memory controllers without compromising efficiency can facilitate better customization of computing systems, which in turn can reduce the overall waste and total cost of building and maintaining these computing systems.

[0032] Example System

[0033] Figure 11 is a block diagram illustrating an example system including one or more computing devices 100. Computing device 100 may include one or more processors 110, memory 120, and one or more communication interfaces 130 for sending and receiving information, such as data or code representing guidelines or instructions for writing, accessing, or processing data. In some examples, computing device 100 may include a single-core or multi-core chip. In some examples, computing device 100 may include a single-core or multi-core chip and additional devices connected to the chip, such as one or more memory devices 124.

[0034] The one or more processors 110 may include a CPU core in communication with the memory 120. The core may be a microprocessor on a chip, a multi-core processor, or any other known processor, such as a GPU, FPGA, ASIC (such as a TPU), etc. Figure 1 Only one box is shown, but any number of CPU cores may be included.

[0035] The memory 120 may include a combination of a memory controller 122, a memory device 124, and a system-level cache 126 for processing incoming memory access requests, such as read or write requests. The memory controller 122 may be configured to communicate with the memory device 124 to access a memory space corresponding to the requested data. The memory device 124 may include any one or more types of memory, such as a read-only memory, a random access memory, a removable storage medium, a cache, a register, etc. For example, the memory device may include a dynamic random access memory (DRAM) module, which may or may not include one or more dual in-line memory modules (DIMMs). Other examples of the memory device 124 may include a high bandwidth memory (HBM). The memory module may be connected to the processor 110 and other components of the computing device 100.

[0036] Figure 2 is a block diagram showing an example arrangement of a memory controller and a memory device. Figure 2 In the example of , a system including 10 memory controllers 210 labeled MC0 to MC9 is shown. In other examples, a different number of memory controllers can be included. For the purposes of this disclosure, it is assumed that the number of memory controllers is not a power of 2, which means that the number of memory controllers is not 2, 4, 8, 16, etc.

[0037] Each memory controller 210 is connected to a corresponding memory device or memory module 220 to access a memory space 230. The memory space 230 itself may be divided into and may include a plurality of regions 235. Figure 2In the example of , there are 20 regions labeled as region 0 through region 19. As described in more detail herein, each region may be mapped from multiple memory devices in the memory devices 220, and by extension, from multiple memory controllers in the memory controllers 210 (MC0-MC9). Thus, more than one memory device 220 may be used to access a particular address range within the memory space 230, even though not all memory devices 220 may be available to access the particular address range.

[0038] In some examples, the arrangement of memory controller 210 between requesting nodes and memory devices 220 may operate as a coherent hub interface (CHI), where the cache of shared memory space 230 is coherent for all requesting nodes.

[0039] Back to Figure 1 In some examples, memory 120 may also include one or more system level cache (SLC) 126 blocks (also referred to as last level cache (LLC)). Figure 1 In the example of FIG. 1 , a single SLC 126 block is shown, but in other examples, multiple SLC blocks may be included in the system 100. Figure 1 In the example of FIG. 1 , the SLC 126 block is shown to include a snoop filter 128 configured to monitor access requests to the shared memory of the memory space layer 230 and maintain consistency between the SLC blocks 126 monitoring the memory space layer 230. In operation, the SLC 126 can receive a memory access request from a requesting node and attempt to service the memory access request. If the requested data does not exist in the SLC 126 or other nodes according to the snoop filter 128, the request can be transmitted to the memory controller 122, 210 corresponding to the memory device 124, 220 that can service the memory access request.

[0040] about Figure 2 The address mapping between the memory controller 210 and the memory space 230 layer is shown. Figure 1The one or more processors 110 may be configured to arrange or control such address mapping between the memory space layer 230 and the memory controller 220. Mapping addresses to memory controllers may be implemented in the following manner: the one or more processors 110 first divide the memory space layer 230 of the memory into a certain number of regions; select a certain number of address bits; and apply a hash function to the selected address bits, followed by a modulus of the hash value, wherein the modulus corresponds to a certain number of memory controllers to which the region addresses are assigned. In some examples, the hash function used to assign memory addresses may be a hash algorithm that is a power of 2. This process may evenly distribute address mappings among memory controllers, particularly when the number of regions of the assigned memory addresses is a power of 2. When the total number of memory controllers is a power of 2, the mapping may be simple because the total number of regions may be selected to be a power of 2 (including 1), and each region may be assigned to all memory controllers. In comparison, when the total number of memory controllers in the memory control layer is not a power of 2, the total number of regions into which the addresses are divided can be selected to be different non-power of 2 numbers to evenly distribute the addresses on the non-power of 2 memory controllers.

[0041] Figure 3 An example of a non-power-of-2 distribution is shown in , where addresses of 20 regions of memory space (labeled 0-19) are mapped to 10 memory controllers (labeled 0-9). Figure 3 In the example of , each region of addresses is mapped to 8 of the 10 memory controllers, where the mapping between memory controllers and regions is indicated by the shaded boxes. For example, region 0 is mapped to memory controllers 0, 1, 2, 3, 5, 6, 7, and 8. For another example, region 1 is mapped to memory controllers 0, 1, 2, 4, 5, 6, 7, and 9. Figure 3 It can be seen that each region 0-19 is mapped to exactly 8 of the 10 controllers 0-9. Furthermore, the mapping is arranged so that each memory controller is mapped to the same number of regions, 16 regions in this example.

[0042] In some examples, a mask algorithm can be used to implement Figure 3The creation of a mapping as shown in , wherein, rather than indicating which regions the memory controller is mapped to, each region that is not mapped to the memory controller is masked for the corresponding memory controller. In other examples, the algorithm can identify specific regions mapped to the memory controller without the need to mask unallocated regions, such as by applying a non-power-of-two hashing algorithm to the address bits of the addresses in each region. In the case of a non-power-of-two hash, the size of the hash table may not be a power of 2, which is known to have the effect of providing a more uniform distribution and reducing conflicts during hashing. In such examples, an address mask can be used to control the granularity of address mappings within a cache of a memory space, which means that instead of mapping alternating cache lines to different regions, multiple cache lines (such as 2 cache lines, 4 cache lines, 8 cache lines, 16 cache lines, 32 cache lines, etc.) can be alternately mapped to different regions.

[0043] The number of regions of memory and the number of regions mapped to each memory controller may be selected by one or more processors 110 based on predefined rules or criteria. One rule is that the number of regions mapped to each memory controller should be a power of 2. In some cases, this number may be selected to be a maximum value that is a power of 2 and is less than the total number of regions of memory space. For example, in Figure 3 In the example of , the total number of regions is 20, and the number of regions mapped to each memory controller is 16, where 16 is the largest power of 2 number that is less than 20. In other cases, smaller values ​​may be used, such as in Figure 3 In the example shown in , 8 or 4 regions are mapped to each memory controller.

[0044] Additionally, in some examples, the number of regions can be equal to the number of memory controllers, ensuring that each memory controller maps to the same number of regions. For example, if there are 5 regions and 5 memory controllers, each of the 5 memory controllers can be mapped to 4 of the 5 regions, making each region exactly accessible to 4 memory controllers. Likewise, in a different example, if there are 15 regions and 15 memory controllers, each of the 15 memory controllers can be mapped to 8 of the 15 regions, making each region exactly accessible to 8 memory controllers. However, if Figure 3 As shown, there are 20 regions and 10 memory controllers, and it is not always required that the total number of regions and the total number of memory controllers be equal to each other.

[0045] exist Figure 1 and Figure 2In the example of , a single group of memory controllers MC0-MC9 is shown as being evenly divided between the memory devices of a single memory space. However, in other example systems, the memory space itself can be subdivided into respective memory space clusters, which may or may not be contiguous, wherein memory addresses of the memory space can be addressed to different memory space clusters in the respective memory space clusters. In such examples, the respective memory space clusters can be controlled by corresponding memory controller clusters, wherein the memory controllers of one memory controller cluster are kept separate from the memory controllers of other memory controller clusters.

[0046] Figure 4 An example system of this type is shown in FIG. Figure 1 and Figure 2 Like the example system, Figure 4 The example system includes one or more computing devices 400 having one or more processors 410, memory 420, and a communication interface 430 for communicating with other systems or system users. Figure 1 and Figure 2 The system memory 120 is the same as that of Figure 4 The memory 420 of the system may include each of a memory controller, a memory device, and a memory space. Figure 4 system and Figure 1 and Figure 2 The difference of the system is that the memory control 440 component and the memory space 450 component are divided into multiple clusters, wherein each of the memory controllers (MC) 445 is mapped to a different corresponding memory controller cluster 4421-442 N , and memory address 455 is mapped to memory controller clusters 4421-442 linked to N The different corresponding memory space clusters 4521-452 of the corresponding memory controller cluster N The mapping may involve using a power-of-two hashing algorithm to assign each of the memory addresses 455 to its different corresponding memory space clusters 4521-452. N .

[0047] although Figure 4 The memory devices are not shown, but it should be understood that the memory control clusters 4421-442 N Each memory controller included in each of the memory devices may communicate with the corresponding memory device to access its memory in the corresponding memory space cluster 4521-452 N Therefore, the memory control cluster 4421-442 NEach pair of memory space clusters 4521-452 N It may also include its own corresponding memory device.

[0048] With respect to clustering memory controllers into memory controller clusters, the number of memory controller clusters may vary from system to system depending on several factors including, but not limited to, the total number of memory controllers, the locations of the memory controllers relative to each other, and the locations of the memory devices included in the memory space relative to each other.

[0049] While in some cases a greater number of memory controllers may be associated with a greater number of memory controller clusters, it should be understood that this is not the only way in which the total number of memory controllers may affect the number of memory controller clusters. For example, the number of memory controllers per memory controller cluster may be selected to facilitate efficient partitioning of the memory controllers into regions, such as in conjunction with Figure 2 As described. For example, if the total number of memory controllers is 54, the number of memory controller clusters can be 3 (18 memory controllers per cluster), 6 (9 memory controllers per cluster), or 9 (6 memory controllers per cluster). In a cluster including 18 memory controllers, the number of regions accessed by the memory controller can be 9 (e.g., each memory controller can access 8 of the 9 regions), or 18 (e.g., each memory controller can access 16 of the 18 regions). In a cluster including 9 memory controllers, the number of regions accessed by the memory controller can be limited to 9 (e.g., each memory controller can access 8 of the 9 regions). In a cluster including 6 memory controllers, the number of regions accessed by the memory controller can be limited to 6 (e.g., each memory controller can access 4 of the 6 regions). Since the preferred total number of regions for a memory space cluster can vary from system to system, the preference for a specific number of memory controllers for each memory controller cluster can also vary from system to system.

[0050] Will be clustered with the corresponding memory controller 4421-442 N Associated memory space cluster 4521-452 N Mapping to memory controller 445 may involve mapping memory space clusters 4521-452 N In a given memory space cluster 4521-452 NIn an example where the size of the memory cluster is equal to a power of 2 (such as 32GB, 64GB, etc.), a hash of the address bits can be used to map a specific address to a given memory cluster. The address bit mask can be used to control the granularity at which the hashing is performed, which means that instead of associating alternating cache lines with different clusters, multiple cache lines (such as 2 cache lines, 4 cache lines, 8 cache lines, 16 cache lines, 32 cache lines, etc.) can be associated with different clusters alternately.

[0051] Example Method

[0052] Figure 5 is a flow chart illustrating an example process 500 for mapping memory addresses to a memory controller according to aspects of the present disclosure. The process 500 may be performed by one or more processors such as Figure 1 It should be understood that in at least some cases, although one or more processors are used to perform the mapping, once the mapping is created, memory access requests can be processed without the one or more processors, such as at the SLC itself.

[0053] At block 510, one or more processors may assign addresses of the memory space to a plurality of regions, referred to as "M" regions. Each address of the memory space may belong to only one region. The number of regions "M" may be based on various factors, such as the size of the memory space, the number of memory controllers (referred to herein as "N") that control the memory space, and the like.

[0054] At box 520, one or more processors may map each of the memory controllers to a subset of the M regions, meaning that those memory controllers mapped to a certain region can access addresses associated with that region. The number of regions mapped to each memory controller (referred to herein as "J") can be the same for each memory controller. In other words, each memory controller can access the same amount of space in the memory space. In addition, it should be understood that although the value of J is necessarily lower than the value of M, the value of J does not have to be a factor of M because each region can be mapped to multiple memory controllers. One requirement for the value of J is that it should be a power of 2 so that the hash of the memory space across the controllers is effectively distributed between the memory controllers to avoid latency issues or locality issues.

[0055] exist Figure 5 In the example of , the process may receive a predetermined number "N" of memory controllers as input 530. In addition, Figure 5In the example of , the number of regions "M" may also be predetermined prior to process 500. However, in other examples, as part of block 510, a value for "M" may be determined. Such a determination may be based on the value of "N" as well as other factors, such as the size of the memory space. Similarly, in Figure 5 In the example of FIG5 , the number “J” of regions to which each memory controller is mapped may also be predetermined prior to process 500. However, in other examples, a value for “J” may be determined as part of block 520. Such a determination may be based on the values ​​of “M” and “N” as well as other factors, such as the size of the memory space.

[0056] The systems and methods described herein can be used to facilitate memory access requests across multiple memory controllers without reducing mesh bandwidth, even if the total number of memory controllers is not a power of 2. This can be particularly advantageous for chip developers seeking ways to increase the number of memory controllers on a chip without having to double the total number. In other words, a chip can be designed with any number of memory controllers, whether or not the number is a power of 2, resulting in greater chip design flexibility without degrading overall chip performance or degrading the performance of certain memory elements of the chip.

[0057] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the previous description of the embodiments should be made by way of illustration rather than by way of limitation of the subject matter defined by the claims. In addition, the examples described herein and the provision of clauses expressed as "such as", "including", etc. should not be interpreted as limiting the subject matter of the claims to specific examples; on the contrary, these examples are intended to illustrate only one of many possible embodiments. In addition, the same reference numerals in different figures may identify the same or similar elements.

Claims

1. A system comprising: a memory space accessible to read or write data; N memory controllers, where N is an integer greater than 1 and not a power of 2; and One or more processors configured to: assigning addresses of the memory space to M regions, where M is a plurality; and Each of the N memory controllers is mapped to a subset of the M regions.

2. The system of claim 1, wherein: Each subset of the M regions includes exactly J regions, where J is less than M and is a power of 2.

3. The system of claim 2, wherein: J is a maximum value that is a power of 2 and less than M.

4. The system of claim 2, wherein: The one or more processors are configured to map more than one memory controller to each of the M regions and to map the same number of memory controllers to each of the M regions.

5. The system of claim 2, wherein: N equals 5, M equals 5, and J equals 4; or N is equal to 6, M is equal to 6, and J is equal to 4.

6. The system of claim 1, wherein: The memory space includes a plurality of system level cache memory blocks associated with a plurality of core processing units.

7. The system of claim 6, wherein: The one or more processors are configured to: dividing both the memory controller and the system-level cache memory into a plurality of non-overlapping clusters; assigning memory addresses to the plurality of clusters; and For at least one cluster, addresses of the memory space of the cluster are assigned to the M regions, and each of the N memory controllers is mapped to a subset of the M regions within the cluster.

8. The system of claim 7, wherein: The one or more processors are configured to: Using a power-of-two hashing algorithm to assign the memory addresses to the clusters; and Within the at least one cluster, determining which system-level cache blocks of the cluster to assign the memory address to using a non-power-of-two hashing algorithm.

9. The system of claim 7, wherein: The one or more processors are configured to control a granularity of cluster alternation of address allocations among the plurality of clusters using an address mask.

10. The system of claim 6, wherein: The N memory controllers are included in a chip.

11. A method comprising: allocating, by one or more processors, addresses of a memory space accessible to read or write data to M regions, where M is a plurality; as well as Each of the N memory controllers is mapped, by the one or more processors, to a subset of the M regions, where N is an integer greater than 1 and not a power of 2.

12. The method of claim 11, wherein: Each subset of the M regions includes exactly J regions, where J is less than M and is a power of 2.

13. The method of claim 12, wherein: J is a maximum value that is a power of 2 and less than M.

14. The method of claim 12, when more than one memory controller is mapped to each of the M regions, and the same number of memory controllers is mapped to each of the M regions.

15. The method of claim 12, wherein: N equals 5, M equals 5, and J equals 4; or N is equal to 6, M is equal to 6, and J is equal to 4.

16. The method of claim 11, wherein: Allocating addresses of the memory space includes allocating system level cache memory blocks to the M regions.

17. The method of claim 16, further comprising: partitioning, by the one or more processors, both the memory controller and the system-level cache into a plurality of non-overlapping clusters; assigning, by the one or more processors, the memory addresses to the plurality of clusters; as well as For at least one cluster, addresses of the memory space of the cluster are assigned, by the one or more processors, to the M regions, and each of the N memory controllers is mapped to a subset of the M regions within the cluster.

18. The method of claim 17, further comprising: assigning, by the one or more processors, the memory addresses to the clusters using a power-of-two hashing algorithm; as well as Within the at least one cluster, determining, by the one or more processors, which system-level cache blocks of the cluster to assign the memory address to using a non-power-of-two hashing algorithm.

19. The method of claim 17, further comprising controlling, by the one or more processors, a granularity of cluster alternation of address allocations among the plurality of clusters using an address mask.

20. The method of claim 17, further comprising using, by the one or more processors, an address mask to control the granularity of address allocation within a cache of the memory space.

Images