Memory controller with command reordering

Abstract

A system (100) for handling requests includes a set of memory banks (109) coupled to a memory controller (103), the memory controller including a set of read queues (105 and 107), the set of read queues including read queues currently designated as the priority read queue (105 or 107). The memory controller (103) loads a read request from an associated processor (101) into the set of read queues (105 and 107). To process the read request, the memory controller (103) is configured to process the read request based on availability of the associated memory bank (109), and if not in the priority read queue (105 or 107), based additionally on whether the read request collides with a recently scheduled read request from the priority read queue (105 or 107). The read requests of the priority read queue (105 or 107) are scheduled. When executing a read request from the priority read queue (105 or 107), if the read request is in front of the priority read queue (105 or 107), the memory controller (103) designates a different read queue of the set of read queues (105 and 107) as the priority read queue (105 or 107).

Description

Technical Field

[0001] Aspects of the present description relate to the field of computing hardware, and in particular, to memory controllers in computing devices. Background Art

[0002] In machine learning applications, common operations require two sources to read from memory and a third source to write the result to memory. In addition, there is usually a direct memory access (DMA) controller that also moves data in or out of the memory pool.

[0003] Because multi-port static random access memory (SRAM) arrays are very expensive, most memory designs that need to service multiple requests in parallel and have a reasonable amount of storage use arrays of individually addressable single-port SRAM banks. In such instances, the memory controller may allow one requester to access a first bank while a second requester simultaneously accesses a second bank. However, performance issues arise when two requesters need to access the same bank at the same time. For example, one approach is to allow one requester to perform access to a given bank, and any other requester attempting to access the same bank will be "stuck" and will not be allowed to proceed until the bank is available.

[0004] Additionally, many machine learning access patterns look like a stream of addresses with a constant offset for each source. When these offsets match distances in address space, driving back to the same group, very poorly performing patterns emerge, with performance overheads of up to 300% or more. To mitigate this, software can add "padding," or wasted memory space, around the information, but this reduces the effective amount of on-chip SRAM.

[0005] When using SRAM banks that can only be accessed once every other cycle, additional problems with address patterns occur, where back-to-back requests end up mapping to the same bank. Such SRAMs are desirable because they are typically smaller in physical area and typically consume less power per access than RAMs that can meet single-cycle access rates, even though they have slower access speeds. Summary of the invention

[0006] The technology described herein includes a memory controller capable of reordering read and / or write requests based on the availability of memory banks to improve the speed and efficiency of data transfer. In various embodiments, the memory controller includes a set of read queues and a control circuit system. The read queues store read requests, and the control circuit system manages the order in which the read requests are processed.

[0007] In one example embodiment, a memory controller is coupled to a set of memory banks and one or more processors. The memory controller loads read requests from the one or more processors into the set of read queues. To schedule the read requests, the memory controller schedules the read request for each read request based on the availability of the associated memory bank and, if the read request is in a read queue that is not currently designated as a priority read queue, based on whether the read request conflicts with a recently scheduled read request from a priority read queue. If the recently scheduled read request of the priority read queue is at the front of the queue, the memory controller designates a different read queue in the set of read queues as a new priority read queue.

[0008] In an example, the memory controller first checks the availability of the previous read request in the queue. If the previous read request is not available, the memory controller avoids scheduling the previous read request and moves to the next read request in the queue. The memory controller continues this process until it can successfully schedule a read request.

[0009] In another example embodiment, the memory controller includes a write queue for storing write requests generated by one or more processors. The memory controller receives write requests from the processors and stores the write requests in the write queue. To process the write requests, for each write request, the memory controller schedules the write request based on the availability of the associated memory bank. If the associated memory bank is not available, the memory controller avoids scheduling the write request and moves to the next write request in the queue. Alternatively, if the associated memory bank is available, the memory controller schedules the write request to be executed.

[0010] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Many aspects of the present specification are provided with reference to the following figures. The components in the figures are not necessarily to scale, but the emphasis is placed on clearly illustrating the principles of the present specification. In addition, in the figures, similar reference numerals indicate corresponding parts throughout the several views. Although several examples are described in conjunction with these figures, the present specification is not limited to the examples described herein. On the contrary, it is intended to cover all alternatives, modifications and equivalents.

[0012] Figure 1 A computing environment with a memory controller in an implementation scheme is shown.

[0013] Figure 2A method of operating a memory controller in an embodiment is shown.

[0014] Figure 3 A memory controller in an implementation is shown.

[0015] Figures 4A-4C The operation sequence in the embodiment is shown.

[0016] Figure 5A-5B The operation sequence in the embodiment is shown.

[0017] Figure 6 A memory controller in an implementation is shown.

[0018] Figure 7 The hashing process in an implementation scheme is shown. DETAILED DESCRIPTION

[0019] Improvements to memory controllers are described herein that improve performance and reduce power consumption, and include other advantages. In various embodiments, the memory controller includes a control circuit system and a plurality of read and write queues. In a scenario focused on a read operation for illustrative purposes, the control circuit system of the memory controller loads a read request into a read queue, and schedules each of the read requests for execution based on the availability of the associated memory bank and whether a given read request is in a queue currently designated as a priority read queue. When executing a read request from the front of the current priority read queue, the queue designated as the priority read queue changes.

[0020] Consider the example of a read request in a non-priority read queue. The control circuit system determines whether the memory bank associated with the read request is available for the read request. If enough cycles have passed since the last time data was read from the memory bank, the memory bank associated with the read request is considered available. If the associated memory bank has enough time to "cool down", it is considered available. Otherwise, it is considered unavailable.

[0021] If the associated memory group is determined to be unavailable, the control circuit system moves to the next read request in the current non-priority read queue so that the process restarts for the next read request (e.g., the control circuit system determines whether the associated memory group of the next read request is available, etc.). However, if the associated memory group for the current read request is determined to be available, the controller continues to determine whether the read request conflicts with the most recently scheduled read request from a different queue in the queue that is currently designated as a priority queue. When the memory group associated with the most recently scheduled read request from the priority read queue is the same as the memory group associated with the current read request in the non-priority read queue, there is a conflict. Therefore, although the target memory group may be in an available state, the target memory group will soon be in an unavailable state because the most recently scheduled read request will be executed soon. If there is no conflict, the control circuit system continues to schedule the current read request for execution. If there is indeed a conflict with the most recently scheduled read request, the control circuit system moves to the next read request in the queue and repeats the analysis described above.

[0022] When a read request is scheduled at the front of a priority read queue, the control circuit system changes the designation of the priority read queue. Therefore, when executing the most recently scheduled read request just described above, if the most recently scheduled read request is already at the front of its queue, the control circuit system changes the priority designation to a different queue in the queue. Otherwise, the priority designation remains the same until the read request at the front of the priority queue has been scheduled. The designation can be changed in a round-robin manner. For example, in the case where there are only two read queues, the priority designation will switch between the two read queues. However, in the case where there are three or more read queues, the priority designation can be changed from one read queue to another read queue in a round-robin manner. Other techniques for changing the priority designation are possible and can be considered within the scope of this specification, such as random (or pseudo-random) designation.

[0023] Consider below the example of the current read request in the queue that is currently designated as the priority read queue. The control circuit system determines whether the memory group associated with the read request is available for the read request. As mentioned, if enough cycles have passed since the last time data was read from the associated memory group and / or data was written to the associated memory group, the memory group associated with the read request is considered to be available. If the associated memory group of the current read request is determined to be unavailable, the control circuit system moves to the next read request in the current queue, and the process restarts (e.g., the control circuit system determines the availability of the associated memory group). In other words, if the associated memory group is unavailable, the control circuit system moves to the next read request in the priority read queue, and repeats this process until the control circuit system can identify a read request with an available memory group in the priority queue. However, if the associated memory group of the current read request is determined to be available, the control circuit system continues to schedule the read request because the read request will not conflict with the most recently scheduled read request from the same queue.

[0024] As mentioned, when a read request is scheduled at the front of the priority read queue, the control circuit system changes the designation of the priority read queue. Therefore, if the current read request is at the front of the current queue, and since it is assumed herein that the current queue is the priority queue, the control circuit system switches the designation of the priority queue to a different queue in the plurality of queues. In either case, the control circuit system proceeds to the next read request in a different read queue in the plurality of read queues.

[0025] In another aspect of the technology described herein, a modified mapping of addresses to memory banks is described that reduces the likelihood of back-to-back accesses to the same bank for access sequences of addresses of increasing constant offsets. The modified mapping solves one of the problems encountered with the prior art with problematic offsets, including allowing designs to use dual-cycle access SRAMs that are more dense and lower power by comparison.

[0026] Also described is an internal memory circuit, or "mem" block, which has a specific number of read and write ports. Each port can have one access service per cycle. An arbitration network for all requesters routes each requester to a specific read or write port. If a requester is capable of generating both a read request and a write request, the read and write traffic will be separated and sent to different ports on the mem block.

[0027] Finally, a command queue is described for each read port that holds multiple commands. In various examples, the queue holds eight commands for each read port. A "preferred" read port is selected based on a round-robin selector. Commands on the preferred port take priority, while commands on non-preferred ports are scanned to find transactions that do not conflict with the preferred port.

[0028] Write ports are also described, and are serviced in the order of the write port queue. However, writes can be dequeued into a write node. There are more write nodes than write ports in the design to allow multiple writes from the same port to be committed simultaneously, which allows the write port to "catch up" when a write is delayed due to a conflict. When a write is in a write node, the write is ready to be scheduled to perform a group access. If there is no read port that needs to access the same group, the write can be dequeued from the write node. If the write port cannot move the transaction to the write node because the write node is full, the write of the write node will take precedence over the read.

[0029] Now turning to the pictures, Figure 1 A computing environment 100 in an embodiment is shown. The computing environment 100 includes a CPU 101, a memory controller 103, and a memory block 109. The CPU 101 is a representative processing resource (e.g., a processor core and / or groups thereof with or without supporting circuitry) that needs to access memory to perform associated functions. For example, the CPU 101 may represent a system that employs a neural network to perform tasks (e.g., object detection, image classification, etc.).

[0030] CPU 101 generates a read request to be handled by memory controller 103. Memory controller 103 is coupled to memory block 109 to handle the read request. In an embodiment, CPU 101 includes a streaming engine that provides the read request to memory controller 103. Memory controller 103 includes a command queue for each streaming engine of CPU 101. In operation, the streaming engine of CPU 101 provides the read request to the corresponding command queue of memory controller 103. In response, memory controller 103 checks the read request of the command queue to determine which read request to execute.

[0031] The memory controller 103 represents a controller configured to schedule and execute read requests based on the availability of read requests. The memory controller 103 may be implemented in a larger context to act as an on-chip mechanism for handling read requests, as will be described later. Figure 3Memory controller 103 is coupled to memory block 109 and includes, but is not limited to, command queue 105 and command queue 107. Memory block 109 represents a memory element including a plurality of memory groups. In the example, memory block 109 has been shown to include a total of eight memory groups, but more groups or fewer groups are possible.

[0032] In an embodiment, memory controller 103 includes circuitry configured to specify priorities between command queues. The priority designation indicates to memory controller 103 which command queue represents a priority command queue. More specifically, the priority designation indicates to memory controller 103 which command queue to check first during an execution cycle. For example, if command queue 105 represents a priority command queue, memory controller 103 will first check command queue 105 for available read requests before checking command queue 107, and thus can give priority to requests present in command queue 105.

[0033] In some examples, each memory bank of the memory block 109 represents an addressable single-port SRAM bank that accepts only one access every other clock cycle, which means that after execution, the SRAM banks of the memory block 109 will enter a cool state before allowing the next execution. For example, if memory bank 7 allows access in the first clock cycle, memory bank 7 will remain unavailable until the third clock cycle because memory bank 7 will be in a cool state during the second clock cycle.

[0034] In an embodiment, the addresses of the SRAM banks are hashed to avoid problematic offsets that are detrimental to the performance of the computing environment 100. For example, in the context of a neural network, when performing read requests, it is common to have access patterns with large strides that repeatedly return to the same subset of addresses. The strides that map a subset of addresses to the same SRAM bank perform very poorly due to every other access cycle of the SRAM bank being employed. In some cases, such access patterns result in up to 300% performance overhead. Such access patterns are avoided when addresses are distributed among the SRAM banks (e.g., by hashing or other address allocation). With respect to Figure 7 A method of hashing the addresses of a memory bank is described in detail. In some examples, the addresses of the SRAM bank are hashed statically before live read / write operations.

[0035] Figure 2A method of operating a memory controller (i.e., memory controller 103) including multiple read queues is shown, referred to herein as method 200. First, the method includes loading read requests into the multiple read queues (step 201), such that one of the multiple read queues is currently designated as a priority read queue. The read request may be generated by a processor associated with the memory controller (i.e., any one of CPUs 101). In an embodiment, the associated processor includes a streaming engine that is configured to provide the read request to the corresponding read queue. For example, in Figure 1 In the context of , the first streaming engine of the CPU 101 provides the read request to the command queue 105 , while the second streaming engine provides the read request to the command queue 107 .

[0036] Next, for at least one of the read requests, the memory controller schedules the read request to be executed based on the availability of the read request (step 203). In some instances, in order for a read request to be considered available within a priority read queue, the associated memory group must also be available. Alternatively, in order for a read request to be available within a non-priority read queue, the memory controller may require that the associated memory group be available and that the read request of the non-priority read queue does not conflict with the most recently scheduled read request of the priority read queue. In operation, the memory controller sequentially scans the read requests of the priority read queue to determine a first read request available for execution. Next, the memory controller sequentially scans the read requests of the non-priority read queue to identify a second read request available for execution. Upon identifying available read requests from the priority read queue and the non-priority read queue, the memory controller schedules the available requests for execution. In an embodiment, the memory controller executes the requests simultaneously. In another embodiment, the memory controller executes the read request of the priority read queue before executing the read request of the non-priority read queue. Figure 1 In the context of , the memory controller 103 first scans the read requests of the priority read queue (e.g., command queue 105) to determine a first read request available for execution. Subsequently, the memory controller 103 scans the read requests of the non-priority command queue (e.g., command queue 107) to determine a second read request available for execution. Upon identifying available requests from each queue, the memory controller 103 schedules and executes the requests.

[0037] If the read request scheduled and executed from the priority read queue is the first read request in the queue (i.e., the previous read request), the method continues by designating a different read queue among the multiple read queues as a priority read queue (step 205). In an embodiment, the memory controller includes a priority pointer that is configured to determine priority designations between the multiple read queues. When the previous read request of the priority read queue has been executed, the priority pointer reassigns the priority designation. In an embodiment, the priority pointer reassigns the priority designation based on the capacity of the read requests within the remaining read queues. For example, the priority pointer may reassign priority to the read queue that contains the most read requests. In another embodiment, the priority pointer reassigns the priority designation to the read queue that has stored its previous read request for the longest amount of time. In Figure 1 In the context of , when the previous read request of the priority command queue (e.g., command queue 105) has been executed, the priority circuitry of the memory controller 103 assigns a priority to a non-priority command queue (e.g., command queue 107). When the previous read request of the new priority command queue has been executed, the priority circuitry returns the priority designation.

[0038] Figure 3 Controller 300 in an implementation is shown. Controller 300 includes, but is not limited to, memory controller 319 (i.e., memory controller 103), memory block 321 (i.e., memory block 109), and an arbitration network. Controller 300 may be coupled to one or more processors that send access requests supporting software applications to controller 300. Examples of software applications running on a processor include, but are not limited to, artificial neural networks and other such machine learning algorithms (although any type of software may be executed on a processor and may benefit from the memory technology described herein). Controller 300 executes the request and returns data to the associated processor. The data may be, for example, program code to be executed by the processor and / or actual data to be processed.

[0039] The arbitration network of controller 300 is configured to receive requests from associated processors and route requests accordingly. The arbitration network of controller 300 includes read routers 301 and 303, routers 305 and 307, read ports 311 and 315, and write ports 315 and 317. The router of the arbitration network receives requests from associated processors and routes the requests to specific read or write ports accordingly. In an embodiment, each router of the arbitration network corresponds to a requester of an associated processor. This means that controller 300 includes a router for each requester of an associated processor. The arbitration network reduces the number of global lines required for controller 300 because the processor no longer needs to directly access each group of memory block 321. In fact, each requester of an associated processor provides a service request to a corresponding router to obtain access to data of memory block 321.

[0040] Read routers 301 and 303 represent components that only manage read requests. In an embodiment, read routers 301 and 303 receive read requests from the corresponding streaming engines of the associated processors. For example, the first streaming engine can deliver the read request to read router 301, and the second streaming engine delivers the read request to read router 303. In response, read router 301 routes the read request to read port 311, and read router 303 routes the read request to read port 313.

[0041] Routers 305 and 307 represent components that manage both read and write requests. In an embodiment, routers 305 and 307 receive requests from corresponding components of an associated processor. For example, a data memory controller (DMC) of an associated processor may deliver a request to router 305, while an extended level 2 cache (EL2) delivers the request to router 307. In response, router 305 routes the request to a read port 311 or a write port 315, depending on the type of request. Similarly, router 307 routes the request to a read port 313 or a write port 317.

[0042] Read ports 311 and 313 represent components configured to load read requests to respective command queues of memory controller 319. In an embodiment, memory controller 319 includes a command queue for each read port of controller 300. In operation, read port 311 and read port 313 load read requests to respective command queues of memory controller 319.

[0043] Write ports 315 and 317 represent components configured to load write requests to respective command queues of memory controller 319. In an embodiment, memory controller 319 includes a command queue for each write port of controller 300. In operation, write port 315 and write port 317 load write requests to respective command queues of memory controller 319.

[0044] Memory controller 319 represents a controller configured to perform service requests. Memory controller 319 includes a command queue for each read and write port of controller 300. The command queue of memory controller 319 stores requests for a particular router. For example, a first command queue coupled to read port 311 stores read requests from read router 301 and router 305. Similarly, a second command queue coupled to read port 313 stores read requests from read router 303 and router 307. Alternatively, a third command queue and a fourth command queue coupled to write ports 315 and 317, respectively, store write requests from the corresponding routers. In an embodiment, memory controller 319 represents Figure 1 , 4A -4C, 5A-5B and the memory controller shown in 6.

[0045] The memory block 321 external to the memory controller 319 represents a memory element including a plurality of memory banks. In an embodiment, each memory bank of the memory block 321 represents an addressable single-port SRAM bank that is accessed only once every other clock cycle. Prior to operation, the address of the SRAM bank is hashed to avoid problematic offsets (see later). Figure 7 In the embodiment, the memory block 321 represents Figure 1 Memory block 109.

[0046] In operation, the associated processor begins generating read and write requests for the router of controller 300. Read requests received by read router 301 or router 305 are sent to read port 311. Alternatively, read requests received by read router 303 or router 307 are sent to read port 313. Write requests received by router 305 are sent to write port 315, while write requests received by router 307 are sent to write port 317.

[0047] Upon receiving a request, each port of controller 300 provides the request to a corresponding command queue. In response, memory controller 319 begins executing the request of the command queue. In an embodiment, memory controller 319 uses method 200 to process a read request, such as Figures 4A-4C In another embodiment, the memory controller 319 uses Figure 5A-5B The write request is processed according to the sequence of operations depicted in . If the request requires, the memory controller 319 will return the requested data to the associated processor.

[0048] Figures 4A-4C 4. The memory controller 401 is a memory controller configured to handle read requests. The memory controller 401 may be implemented in a larger context to act as an on-chip mechanism for handling read requests. For example, the memory controller 401 may represent the memory controller 319 of the controller 300. In such a context, the memory controller 401 receives read requests from the read port 311 and the read port 313. The memory controller 401 is coupled to the memory block 409 and includes, but is not limited to, a priority pointer 403, a command queue 405, and a command queue 407. In other embodiments, the memory controller 401 includes more than two command queues, but for the purpose of explanation, only two command queues will be described herein.

[0049] Priority pointer 403 may represent hardware, software, firmware, or a combination thereof configured to assign priorities between command queues of memory controller 401. The priority designation assigned by priority pointer 403 provides an indication of which command queue to service first by memory controller 401. In operation, priority pointer 403 reassigns priority designations when a previous read request of a priority command queue has been executed. For example, priority pointer 403 reassigns priority from command queue 405 to command queue 407 when a previous read request of command queue 405 has been executed.

[0050] Command queue 405 and command queue 407 represent queues configured to store read requests. Command queues 405 and 407 may receive read requests indirectly from a processor associated with memory controller 401. For example, a first streaming engine of the associated processor may be provided for read requests applied to command queue 405, while a second streaming engine of the associated processor may be provided for read requests applied to command queue 407. Figure 3 In the context of , command queue 405 may receive read requests from read port 311, while command queue 407 receives read requests from read port 313. In an embodiment, memory controller 401 includes a command queue for each read requester (ie, streaming engine) of the associated processor.

[0051] In an embodiment, the read requests of command queues 405 and 407 specify a desired memory bank for executing the read request. For example, the currently scheduled read request of command queue 405 attempts to access memory bank 0, while the currently scheduled read request of command queue 407 attempts to access memory bank 7. Memory controller 401 determines which read request to schedule based on the availability of the associated memory bank and the current priority designation.

[0052] Memory block 409 represents a memory element (e.g., memory block 109) that includes multiple memory groups arranged in parallel. In an embodiment, each memory group of memory block 409 represents an addressable single-port SRAM group that accepts only one access every other clock cycle. This means that after execution, the SRAM groups of memory block 409 require a cool-down cycle to allow the next execution. Memory groups 1, 2, and 5 represent SRAM groups that were accessed in the previous cycle and therefore require a cool-down cycle before allowing the next execution. Alternatively, the remaining groups of memory block 409 (i.e., memory groups 0, 3, 4, 6, and 7) remain available. In an embodiment, the addresses of the SRAM groups are hashed to avoid problematic offsets that are detrimental to the performance of memory controller 401, referred to later Figure 7 Give a description.

[0053] Now turn to Figure 4A , stage 400A depicts the first execution of a read request. First, the memory controller 401 first identifies the current priority designation indicated by the priority pointer 403. Next, the memory controller 401 begins scanning the priority command queue (i.e., command queue 405) to determine the first read request available for execution. This means that the memory controller 401 identifies the first read request in the queue that has an available memory group. Starting at the front of the command queue 405, the memory controller 401 checks the availability of memory group 2. Since memory group 2 is in a cool state, the memory controller 401 moves to the next read request in the command queue 405 and checks the availability of the corresponding memory group. Since memory group 0 is available for access, the memory controller 401 schedules the corresponding read request. Because the scheduled read request is not at the front of the queue, the command queue 405 maintains the priority designation.

[0054] When scheduling a read request for command queue 405, memory controller 401 starts scanning command queue 407 to determine the first read request available for execution from the queue. This means that memory controller 401 identifies the first read request with an available memory group and does not interfere with the recently scheduled read request of the priority command queue. Starting in front of command queue 407, memory controller 401 checks the availability of the previous read request. Since the previous read request conflicts with the recently scheduled read request of command queue 405, memory controller 401 moves to the next read request in command queue 407 and checks the availability of the next read request. Since memory group 2 is in a cooling state, memory controller 401 moves to the next read request in command queue 407 and checks the availability. Since memory group 7 is available for access and the corresponding read request does not conflict with the recently scheduled read request of command queue 405, memory controller 401 schedules the read request. Therefore, memory controller 401 executes the scheduled read request and returns the result to the corresponding requester. In an embodiment, memory controller 401 executes the request simultaneously. In another embodiment, the memory controller 401 executes read requests of the priority read queue before executing requests of the non-priority read queue.

[0055] Now turn to Figure 4B , stage 400B depicts the next execution of memory controller 401, such that the next execution is after the execution depicted in stage 400A. As a result of stage 400A, command queues 405 and 407 receive new read requests to be stored. In addition, memory block 409 now depicts memory banks 1, 2, and 5 as available, and memory banks 0 and 7 as being in a cool state.

[0056] At the completion of stage 400A, the memory controller 401 rechecks the priority assignments of the command queue. Since the priority assignments have not changed from those shown in stage 400A, the memory controller 401 begins scanning the command queue 405 to determine the first read request in the queue that is available for execution. Starting at the front of the command queue 405, the memory controller 401 checks the availability of memory bank 2. Since memory bank 2 now remains available for access, the memory controller 401 schedules the corresponding read request.

[0057] When scheduling the front read request of the command queue 405, the memory controller 401 begins to scan the command queue 407 to determine the first read request available for execution in the queue. Starting at the front of the command queue 407, the memory controller 401 checks the availability of the front read request. Since the memory group 0 is in a cool state, the memory controller 401 moves to the next read request in the command queue 407 and checks the availability of the request. Since the next read request conflicts with the most recently scheduled read request of the command queue 405, the memory controller 401 moves to the next read request in the command queue 407 and checks its availability. Since the memory group 3 is available for access and the corresponding read request does not conflict with the most recently scheduled read request of the command queue 405, the memory controller 401 schedules the read request. Therefore, the memory controller 401 executes the scheduled read request and returns the result to the corresponding requester. In addition, when the front read request of the command queue 405 has been executed, the priority pointer 403 switches the priority designation to the command queue 407.

[0058] Now turn to Figure 4C , stage 400C depicts the next execution of memory controller 401, such that the next execution is after the execution depicted in stage 400B. As a result of stage 400B, command queues 405 and 407 receive new read requests, and memory block 409 depicts memory banks 0 and 7 as available, and memory banks 2 and 3 as being in a cool state.

[0059] First, the memory controller 401 identifies the current priority designation indicated by the priority pointer 403. Upon completion of stage 400B, the priority pointer 403 transfers the priority designation from the command queue 405 to the command queue 407. Thus, when determining the first read request from the queue available for execution, the memory controller 401 identifies the first read request of the command queue 407 with an available memory bank. Starting at the front of the command queue 407, the memory controller 401 checks the availability of memory bank 0. Since memory bank 0 now remains available for access, the memory controller 401 schedules the corresponding read request.

[0060] While scheduling the previous read request of command queue 407, memory controller 401 begins scanning command queue 405 to determine the first read request available for execution from command queue 405. Since command queue 405 no longer maintains a priority designation, when determining the first read request available for execution, memory controller 401 identifies the first read request of command queue 405 that has an available memory bank and does not interfere with the most recently scheduled read request of the priority command queue (i.e., command queue 407).

[0061] Starting at the front of the command queue 405, the memory controller 401 checks the availability of the previous read request. Since the memory group 5 is available for access and the corresponding read request does not conflict with the most recently scheduled read request of the command queue 407, the memory controller 401 schedules the read request. Therefore, the memory controller 401 executes the scheduled read request and returns the result to the corresponding requester. In addition, when the previous read request of the command queue 407 has been executed, the priority pointer 403 switches the priority designation back to the command queue 405. The priority designation changes only when the previous read request of the priority command queue is executed. This means that the execution of the previous read request from the non-priority command queue does not affect the priority designation.

[0062] Figure 5A and 5B 5. The memory controller 501 is shown as a memory controller configured to handle write requests. The memory controller 501 may be implemented in a larger context to serve as an on-chip mechanism for handling write requests. For example, the memory controller 501 may represent the memory controller 319 of the controller 300. In such a context, the memory controller 501 receives write requests from the write port 315 and the write port 317. Although only two write ports (315 and 317) are shown, it is understood that more than two write ports are possible. In an embodiment, the memory controller 501 is further configured to handle read requests. For example, the memory controller 501 may represent the memory controller 103 or the memory controller 401, so that the elements for handling read requests are not shown.

[0063] Memory controller 501 includes, but is not limited to, command queue 503, command queue 505, and write nodes 507, 511, 515, and 519. Although only two command queues (503 and 505) are shown, it can be appreciated that more than two command queues are possible. For example, memory controller 501 may include a command queue for each write port of an associated device.

[0064] Command queue 503 and command queue 505 represent queues configured to store write requests. In operation, command queues 503 and 505 store write requests in the order in which write requests are received. Command queues 503 and 505 can indirectly receive write requests from a processor associated with memory controller 501. For example, the DMC of the associated processor can be applied to the write request of command queue 503, and the EL2 of the associated processor supplies the write request for command queue 505. In an embodiment, command queues 503 and 505 receive write requests from the corresponding ports (i.e., write port 315 and write port 317) of the associated device (controller 300). Command queues 503 and 505 provide write requests to write nodes 507, 511, 515 and 519. The write requests stored by command queues 503 and 505 specify the desired write nodes for executing the requests.

[0065] Write node 507 represents "write node 0", and is coupled to memory group 523 and memory group 525. Write node 511 represents "write node 1", and is coupled to memory group 527 and memory group 529. Write node 515 represents "write node 2", and is coupled to memory group 531 and memory group 533. Write node 519 represents "write node 3", and is coupled to memory group 535 and memory group 537.

[0066] In an embodiment, each of write nodes 507, 511, 515, and 519 includes a queue for each command queue of memory controller 501. In operation, command queues 503 and 505 provide write requests to the corresponding queues of appropriate write nodes. Appropriate write nodes describe write nodes that are coupled to the correct memory group for executing write requests. For example, if command queue 503 stores a write request to memory group 527, command queue 503 will provide the write request to queue 512 of write node 511. Alternatively, if command queue 505 stores a write request to memory group 527, command queue 505 will provide the write request to queue 513 of write node 511. More specifically, command queue 503 may provide the write request to queue 508 of write node 507, queue 512 of write node 511, queue 516 of write node 515, or queue 520 of write node 519 according to the requirements of the write request. Alternatively, command queue 505 may provide write requests to queue 509 of write node 507, queue 513 of write node 511, queue 517 of write node 515, or queue 521 of write node 519. There are more write nodes than command queues to allow multiple write transactions from the same command queue to be executed simultaneously.

[0067] In an embodiment, memory banks 523, 525, 527, 529, 531, 533, 535, and 537 represent addressable single-port SRAM banks that accept only one access every other clock cycle. This means that memory banks 523, 525, 527, 529, 531, 533, 535, and 537 require a cool-down cycle before allowing the next execution. Prior to operation, the addresses of the memory banks are hashed to avoid problematic offsets, which are later Figure 7 Described in.

[0068] Now turn to Figure 5A , stage 500A depicts the first execution of a write request. First, command queues 503 and 505 receive write requests from associated devices. Next, memory controller 501 dequeues the previous write requests of command queues 503 and 505 to the corresponding queues of the appropriate write nodes. In response, memory controller 501 determines the availability of write requests stored by the queues of the write nodes. The write request can only be executed if the corresponding memory bank is available. Unavailable memory banks can serve different requests or cool down for the next execution.

[0069] In an embodiment, the memory controller 501 serves both read requests and write requests. In some instances, by default, a read request takes precedence over a write request. This means that when a read request and a write request need to access the same memory bank, the memory controller 501 will execute a read request before executing a write request. Thus, when the associated memory bank is serving a read request, a write request may not be executed. In an embodiment, when the queue of the write node is full, the memory controller 501 gives priority to a write request over a read request (pointing to the same memory bank). This means that the memory controller 501 will execute a write request of a full queue before executing any read request that needs to access the same memory bank as the write request in the full queue. Once the queue is no longer full, a read request takes precedence over a write request.

[0070] When checking the write node, the memory controller 501 determines the following. First, the write request of the queue 508 of the write node 507 is available for execution. Next, the write request of the queue 513 of the write node 511 is not available for execution because the corresponding memory group is currently servicing a different request (i.e., a read request). Finally, the write request of the queue 516 of the write node 515 is not available for execution because the corresponding memory group is in a cool state. Therefore, the memory controller 501 executes the write request of the queue 508. When executing the request, the command queues 503 and 505 receive new write requests from the associated devices, and the memory controller 501 dequeues the previous write requests of the command queues 503 and 505 to the corresponding queues of the appropriate nodes.

[0071] Now turn to Figure 5B , stage 500B depicts the next execution of memory controller 501. At the completion of stage 500A, memory controller 501 determines the availability of write requests stored by the queues of the write nodes. When checking the write nodes, memory controller 501 determines that the write requests of queue 513 of write node 511 are not available for execution because the corresponding memory bank is in a cool state. In addition, memory controller 501 determines that the write requests of queue 516 and queue 517 of write node 511 are available for execution.

[0072] In an embodiment, to determine which write request to execute from the available options, memory controller 501 checks the current capacity of the queues. For example, since queue 516 contains more write requests than queue 517, memory controller 501 will execute the command from queue 516. In another embodiment, memory controller 501 checks the urgency of the request to determine which write request to execute from the available options.

[0073] When determining which request to execute, the memory controller 501 executes one of the available options. Therefore, the command queues 503 and 505 receive the new write request, and the memory controller 501 provides the previous write request of the command queues 503 and 505 to the appropriate queue of the corresponding node. For the remaining write request execution, the sequence of operations depicted in the stages 500A and 500B is repeated.

[0074] Figure 6 Shown for Figure 5A and 5B Another architecture of the memory controller 501 of the present invention is referred to herein as architecture 600. Architecture 600 includes a write request pipeline network. The write pipeline network of architecture 600 is inserted between the command queue and the write node of the memory controller 501, and is configured to route the write request from the command queue to the corresponding queue of the appropriate write node. The write pipeline network of the memory controller 501 includes pipeline 603, pipeline 605, pipeline 607, and pipeline 609. In an embodiment, the write request pipeline network includes a pipeline for each write node of the memory controller.

[0075] Pipeline 603 represents a network configured to load write requests from command queue 503 to corresponding queues of write nodes 507 and 511. Pipeline 603 may load write requests to queue 508 (not shown) of write node 507 or queue 512 (not shown) of write node 511, depending on the requirements of the write request. To determine which write node to provide the request to, pipeline 603 examines the memory bank requirements of the write request.

[0076] Pipeline 605 represents a network configured to load write requests from command queue 505 to corresponding queues of write nodes 507 and 511. Pipeline 605 may provide write requests to queue 509 (not shown) of write node 507 or queue 513 (not shown) of write node 511. When providing a request, pipeline 605 checks the memory bank requirement of the write request to determine which write node to provide the request to.

[0077] Pipeline 607 represents a network configured to load write requests from command queue 503 to corresponding queues of write nodes 515 and 519. Pipeline 607 may provide write requests to queue 516 (not shown) of write node 515 or queue 520 (not shown) of write node 519. Pipeline 607 examines the memory bank requirement of the write request to determine which write node to provide the request to.

[0078] Pipeline 609 represents a network configured to load write requests from command queue 505 to corresponding queues of write nodes 515 and 519. Pipeline 607 may provide write requests to queue 517 (not shown) of write node 515 or queue 521 (not shown) of write node 519. To determine which write node to provide the request to, pipeline 609 examines the memory bank requirements of the write request.

[0079] Figure 7 A hashing process 700 is shown for a memory controller (ie, memory controller 103). Hashing process 700 represents a method for identifying an associated memory bank (ie, a memory bank of memory block 109) that is the subject of a read request or a write request.

[0080] More generally, the memory controllers contemplated herein may operate in one of two modes to identify an associated memory bank: in one mode, the portion of the address identified in the request is used directly to confirm the identity of the memory bank, or in another mode, a hashing algorithm is applied to the address to determine which bank to map the address to. The hashing process 700 describes the latter mode: an algorithm for hashing an address identified in a read or write address to determine its associated memory bank.

[0081] When the benefits of the write process described herein are needed (in order to avoid systematic group conflicts), a suitable hash algorithm is used for write requests. When the data being sought is written using a hash algorithm, a suitable hash algorithm is used for read requests. In other words, if a write request that writes data to its current location in memory is subjected to a hash algorithm, a read request seeking the same data will be subjected to the same hash algorithm. The steps performed by the memory controller when undergoing the hash process 700 are described below.

[0082] In operation, the memory controller receives a service request identifying an address in the memory. As described above, the availability of the memory bank associated with the service request affects whether the request will be scheduled. In order to identify the associated memory bank, the memory controller first multiplies the upper set of addresses by five (step 701), so that the upper set of addresses describes the bits of the address that are larger than the bits describing the byte offset within the bank. For example, if the memory bank is 16 bytes wide, each bit of the address greater than four will be multiplied by five. In an embodiment, the upper set undergoes a shift operation and an addition operation to obtain the desired result.

[0083] Next, the memory controller divides the resulting address into a discrete number of bit fields (step 703). In an embodiment, the number of bit fields is equal to the binary logarithm of the number of bits in the address after the multiplication operation divided by the number of memory banks in the system. For example, upon completion of step 701, if the address now has 12 bits in a system with 8 memory banks, the number of bit fields will be equal to the binary logarithm of 12 divided by 8.

[0084] When dividing the address, the memory controller rotates the multiple bit fields of the address based on the position of the multiple bit fields within the address (step 705). Therefore, the address now contains a discrete number of rotated bit fields. Finally, the memory controller performs an exclusive negative OR (XNOR) operation on the multiple rotated bit fields within the address (step 707). As a result of the XNOR operation, the memory controller has the identification of the memory bank associated with the service request and can proceed to determine the availability of the memory bank.

[0085] Alternative hashing processes are possible. For example, a memory controller may XOR all address bits by dividing the address into N-bit fields and XORing the fields together to produce the final group number (where N is the number of bits required to encode the group number). For example, for a 16-group memory system with 16-byte wide groups, N=4, and the function effectively XORs together all but the lowest nibble of the address, which encodes the byte address within the group. With this technique, any address sequence with a stride that is a power of 2 is guaranteed to reach a different group in subsequent cycles. For example, this occurs for a 4MB (22 address bits), 16-group system with a stride of 0x400 (1KB). However, a disadvantage of such an alternative hashing function is that if an address sequence has the same number of strides in two of the N-bit fields, the resulting group address will periodically end up reaching the same group within 2^N cycles.

[0086] As will be appreciated by those skilled in the art, aspects of the present invention may be embodied as a system, method, or computer program product. Thus, aspects of the present invention may take the form of an entirely hardware implementation, an entirely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects (all of which may be collectively referred to herein as "circuits," "modules," or "systems"). Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0087] In fact, the included descriptions and figures depict specific embodiments to teach those skilled in the art how to make and use the best mode. For the purpose of teaching the principles of the invention, some aspects have been simplified or omitted. Those skilled in the art will understand variations within the scope of this specification from these embodiments. Those skilled in the art will also understand that the features described above can be combined in various ways to form multiple embodiments. Therefore, the present invention is not limited to the specific embodiments described above, but is limited only by the claims and their equivalents.

[0088] The above description and the associated figures teach the best mode of the present invention. The appended claims specify the scope of the present invention. It should be noted that some aspects of the best mode may not fall within the scope of the present invention specified by the claims. Those skilled in the art will appreciate that the features described above can be combined in various ways to form multiple variations of the present invention. Therefore, the present invention is not limited to the specific examples described above, but is only limited by the appended claims and their equivalents.

Claims

1. A method comprising: storing a set of read requests into a set of read queues, wherein the set of read queues includes a first read queue currently designated as a priority read queue; for a first read request in the set of read requests: determining whether to schedule the first read request based on availability of an associated memory bank and if the first read request is in a queue in the set of read queues that is not in the priority read queue, and further based on whether the first read request conflicts with a second read request from the priority read queue; as well as Based on executing the second read request in the priority read queue, determining whether to designate a different read queue in the set of read queues as the priority read queue is determined based on whether the second read request is ahead of the priority read queue.

2. The method according to claim 1, further comprising: Upon determining that the associated memory bank is available, and the first read request is not in the priority read queue and does not conflict with the second read request, then scheduling the first read request to be executed.

3. The method according to claim 1, further comprising: Upon determining that the associated memory bank is available, and the first read request is in the priority read queue, then scheduling the first read request to be executed.

4. The method of claim 1, wherein determining whether the associated memory bank is available is based on whether more than one cycle has occurred since a most recent access to the associated memory bank.

5. The method of claim 1, wherein the first read request conflicts with the second read request when the associated memory bank for the first read request is the same as a memory bank associated with a most recently scheduled read request.

6. The method according to claim 1, further comprising: If the associated memory bank is not available, then refraining from scheduling the first read request and moving to the next read request in the queue, regardless of whether the queue is the priority read queue.

7. The method according to claim 1, further comprising: When scheduling the first read request for execution, moving to a next read request in a different read queue in the set of read queues, regardless of whether the queue is the priority read queue.

8. The method according to claim 1, further comprising: Load write requests into multiple write queues; as well as The write requests are processed, including by, for each write request, dequeuing the write request to a write node coupled to an associated memory bank, and scheduling the write request for execution based on availability of the associated memory bank.

9. The method of claim 8, wherein scheduling the write request to be executed based on the availability of the associated memory bank comprises: When it is determined that the associated memory bank is unavailable and the write node is not full, avoiding scheduling the write request and moving to a next write request in a different write node; as well as upon determining that the associated memory bank is unavailable and the write node is full, prioritizing the write request over a read request directed to the associated memory bank; as well as Upon determining that the associated memory bank is available, the write request is scheduled to be executed.

10. A memory controller comprising: a set of read queues, wherein the set of read queues includes a first read queue currently designated as a priority read queue; as well as a control circuit system coupled to the set of read queues, the control circuit system being configured to: Loading a read request into the read queue set; for at least one of the read requests: scheduling the read request for execution based on availability of an associated memory bank and, if the read request is in a queue that is not the priority read queue, additionally based on whether the read request conflicts with a most recently scheduled read request from the priority read queue; as well as A different read queue in the set of read queues is designated as the priority read queue if the most recently scheduled read request of the priority read queue is ahead of the priority read queue when executing the most recently scheduled read request of the priority read queue.

11. The memory controller according to claim 10, wherein: Upon determining that the associated memory bank is available and the read request is not in the priority read queue and does not conflict with the most recently scheduled read request, the control circuitry schedules the read request for execution.

12. The memory controller according to claim 10, wherein: Upon determining that the associated memory bank is available, and the read request is in the priority read queue, then the control circuitry schedules the read request to be executed.

13. The memory controller according to claim 10, wherein: In order for the associated memory bank to be available, more than one request cycle must have occurred since the most recent access to the associated memory bank.

14. The memory controller of claim 10, wherein the read request conflicts with the most recently scheduled read request when the associated memory bank for the read request and a memory bank associated with the most recently scheduled read request are the same.

15. The memory controller of claim 10, further comprising: Upon determining that the associated memory bank is unavailable, the control circuitry refrains from scheduling the read request and moves to a next read request in the queue, regardless of whether the queue is the priority read queue.

16. The memory controller according to claim 10, wherein: When scheduling the read request for execution, the control circuitry moves to a next read request in a different read queue in the set of read queues, regardless of whether the queue is the priority read queue.

17. A computing device comprising: Storage group collection; a set of processors coupled to the set of memory banks; as well as a memory controller coupled to the set of memory banks and to the one or more processors, wherein the memory controller includes a set of read queues, and wherein the memory controller is configured to: loading read requests from the one or more processors into the set of read queues, wherein the set of read queues includes a read queue currently designated as a priority read queue; for at least one of the read requests: scheduling the read request for execution based on availability of an associated memory bank and, if the read request is in a queue that is not the priority read queue, additionally based on whether the read request conflicts with a most recently scheduled read request from the priority read queue; as well as A different read queue in the set of read queues is designated as the priority read queue if the most recently scheduled read request of the priority read queue is ahead of the priority read queue when executing the most recently scheduled read request of the priority read queue.

18. The computing device of claim 17, wherein the memory controller is configured to: Upon determining that the associated memory bank is available and the read request is not in the priority read queue and does not conflict with the most recently scheduled read request, scheduling the read request for execution and moving to the next request in the priority read queue; Upon determining that the associated memory bank is available and the read request is in the priority read queue, scheduling the read request to be executed and moving to a next request in a different read queue in the set of read queues; as well as Upon determining that the associated memory bank is unavailable, refraining from scheduling the read request and moving to the next read request in the queue, regardless of whether the queue is the priority read queue.

19. The computing device of claim 17, wherein: In order for the associated memory bank to be available, more than one request cycle must have occurred since the most recent access to the associated memory bank.

20. The computing device of claim 17, wherein the read request conflicts with the most recently scheduled read request when the associated memory bank for the read request and a memory bank associated with the most recently scheduled read request are the same.

Images