Memory Controller with Command Reordering

Abstract

A system (100) for processing requests includes a set of memory banks (109) coupled to a memory controller (103) that includes a set of read queues including the read queue currently designated as the priority read queue (105 or 107). The memory controller loads read requests into the set of read queues from the associated processor (101). To process a read request, the memory controller is configured to schedule the read requests of the priority read queue based on the availability of the associated memory banks and based on whether the read request, if not in the priority read queue, conflicts with the most recently scheduled read request from the priority read queue. When executing a read request from the priority read queue, if the read request was at the head of the priority read queue, the memory controller designates a different one of the set of read queues as the priority read queue.

Description

Technical Field

[0001] The described aspects relate to the field of computing hardware, and more particularly, to memory controllers within computing devices.

Background Art

[0002] In machine learning applications, a typical operation requires two sources for reading from memory and a third source for writing results to memory. There is also often a direct memory access (DMA) controller for moving data to and from a memory pool.

[0003] Because multi-port static random access memory (SRAM) arrays are very expensive, in many memory designs that need to process multiple requests in parallel and have a reasonable amount of storage, an array of individually addressable single-port SRAM banks is employed. In such an example, a memory controller may permit one requester to access a first bank while at the same time a second requester accesses a second bank. However, performance issues occur when two requesters need to access the same bank simultaneously. For example, in one approach, one requester is permitted to perform an access to a given bank, and any other requester attempting to access the same bank is "stalled" and not permitted to proceed until that bank becomes available.

[0004] Also, many machine learning access patterns appear as a stream of addresses with a fixed offset for each source. When these offsets match distances within the address space, it results in patterns that return to the same bank and have very poor performance with a performance overhead of up to 300% or more. To mitigate this, software can add "padding" or wasted memory space around that information, which reduces the effective amount of on-chip SRAM.

[0005] An additional problem with address patterns where consecutive (back-to-back) requests map to the same bank occurs when using SRAM banks that accept only one access every other cycle. Such SRAMs are generally smaller in physical area than RAMs that can meet a single-cycle access rate, even with a slow access speed, and generally have low power consumption per access, so they are desirable. SUMMARY OF THE INVENTION

[0006] The technology described herein includes a memory controller capable of rearranging read requests and / or write requests based on the availability of memory banks to improve the speed and efficiency of data transfer. In various implementations, the memory controller includes a set of read queues and control circuit elements. The read queues store read requests, and the control circuit elements control the order in which the read requests are processed.

[0007] In one exemplary implementation, a memory controller is coupled to a set of memory banks and one or more processors. The memory controller loads read requests from one or more processors into a set of read queues. To schedule the read requests, for each read request, the memory controller schedules the read request based on the availability of the associated memory bank and, if the read request is in a read queue that is not currently designated as the priority read queue, also based on whether the read request conflicts with the most recently scheduled read request from the priority read queue. If the most recently scheduled read request in the priority read queue is at the head of the queue, the memory controller designates a different read queue from the set of read queues as the new priority read queue.

[0008] In one example, the memory controller first checks the availability of the head read request in the queue. If the head read request is not available, the memory controller refrains from scheduling the head read request and moves to the next read request in the queue. The memory controller continues this process until the read request can be successfully scheduled.

[0009] In another exemplary implementation, a memory controller includes a write queue for storing write requests generated by one or more processors. The memory controller receives write requests from the processors and stores the write requests in the write queue. To process the write requests, for each write request, the memory controller schedules the write request based on the availability of the associated memory bank. If the associated memory bank is not available, the memory controller refrains from scheduling the write request and moves to the next write request in the queue. Alternatively, if the associated memory bank is available, the memory controller schedules the write request to be executed.

[0010] This summary is provided to introduce, in a simplified form, a series of concepts that are further described in the detailed description below. This summary is not intended to characterize the key features or essential characteristics of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. **Brief Description of the Drawings**

[0011] Many aspects of the present description are provided with reference to the following drawings. In the drawings, components are not necessarily to scale, and emphasis is placed on clearly showing the principles of the description. Also in the drawings, like reference numerals refer to corresponding parts throughout several views. Although several examples are described in connection with these drawings, the description is not limited to the examples described herein. Rather, the intention is to cover all alternatives, modifications, and equivalents.

[0012]

Figure 1

[0013]

Figure 2

[0014]

Figure 3

[0015]

Figure 4A

Figure 4B

Figure 4C

[0016]

Figure 5A

Figure 5B

[0017]

Figure 6

[0018]

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0019] Improvements to the memory controller are described herein as improving performance and reducing power consumption, among other advantages. In various implementations, the memory controller includes control circuit elements and a plurality of read and write queues. In a scenario focused on the read operation for illustration purposes, the control circuit elements of the memory controller load read requests into the read queues and schedule the execution of each of them based on the availability of the associated memory banks and whether a given read request is within the queue currently designated as the priority read queue. When the topmost read request from the current priority read queue is executed, the queue designated as the priority read queue is changed.

[0020] Consider an example of a read request in a non-priority read queue below. The control circuit elements determine whether the memory bank associated with the read request is available for the read request. If sufficient cycles have elapsed since data was last read from the memory bank, the memory bank associated with the read request is considered available. If the associated memory bank had time for sufficient "cool-down", it is considered available. Otherwise, it is considered unavailable.

[0021] If it is determined that the associated memory bank is unavailable, the control circuit element moves to the next read request in the current non-priority read queue, and the process is repeated for the next read request (e.g., the control circuit element determines whether its associated memory bank is available or not). However, if it is determined that the associated memory bank is available for the current read request, the controller proceeds to determine whether the read request conflicts with the most recently scheduled read request from a different one of the queues currently designated as the priority queue. A conflict exists when the memory bank associated with the most recently scheduled read request from the priority read queue is the same as the memory bank associated with the current read request in the non-priority read queue. Therefore, while the target memory bank can be in an available state, the most recently scheduled read request is executed immediately, so that memory bank immediately becomes unavailable. If no conflict exists, the control circuit element proceeds to schedule the current read request for execution. If there is no conflict with the most recently scheduled read request, the control circuit element moves to the next read request in the queue and repeats the above analysis.

[0022] When the read request at the head of the priority read queue is scheduled, the control circuit element changes the designation of the priority read queue. Therefore, when the above-mentioned most recently scheduled read request is executed, the control circuit element changes the priority designation to a different one of the queues if the most recently scheduled read request was at the head of that queue. Otherwise, the priority designation remains the same until a read request at the head of the priority queue is scheduled. The designation can be changed in a round-robin fashion. For example, in an example where there are only two read queues, the priority designation can alternate between them. However, if there are three or more read queues, the priority designation can be changed periodically from one to another. Other techniques for changing the priority designation, such as random (or pseudo-random) designation, are possible and can be considered within the scope of this specification.

[0023] Consider an example of a current read request in the queue currently designated as the priority read queue. The control circuit element determines whether the memory bank associated with the read request is available for the read request. As described above, the memory bank associated with the read request is considered available if a sufficient number of cycles have elapsed since the data was last read from and / or written to the associated memory bank. If it is determined that the memory bank associated with the current read request is unavailable, the control circuit element moves to the next read request in the current queue, and the process is repeated (e.g., the control circuit element determines the availability of the associated memory bank). That is, if the associated memory bank is unavailable, the control circuit element moves to the next read request in the priority read queue and repeats this process until it identifies a read request in a priority queue with an available memory bank. However, if it is determined that the memory bank associated with the current read request is available, the control circuit element proceeds to schedule the read request since it does not conflict with the most recently scheduled read request from the same queue.

[0024] As described above, when the read request at the head of the priority read queue is scheduled, the control circuit element changes the designation of the priority read queue. Therefore, if the current read request is at the head of the current queue, assuming that the current queue is the priority queue here, the control circuit element switches the designation of the priority queue to a different one of the multiple queues. In either case, the control circuit element proceeds to the next read request in a different one of the multiple read queues.

[0025] In another aspect of the technology described herein, address modification mapping for a memory bank is described. The modification mapping reduces the likelihood of consecutive accesses to the same bank for an access sequence of addresses that increment by a fixed offset. The modification mapping solves one of the offset problems that prior art has encountered. This includes enabling a design that uses 2-cycle access SRAM, which has higher density and lower power consumption when compared.

[0026] An internal memory circuit or "mem" block with a specific number of read and write ports is also described. Each port can have an access processed per cycle. A arbitration network for all requesters routes each requester to a specific read or write port. When a requester can generate both read and write requests, the read traffic and write traffic are separated and sent to different ports on the mem block.

[0027] Finally, a command queue for each read port that holds multiple commands is described. In various examples, the command queue holds 8 commands per read port. A "prefferred" read port is selected based on a round-robin style selector. Commands on the preferred port are prioritized and commands on non-preferred ports are scanned to find transactions that do not conflict with the preferred port.

[0028] The write ports that are processed in order from the write port queue will also be described. However, writes can be dequeued to the write nodes. In a design that allows multiple writes to be performed simultaneously from the same port, there are more write nodes than write ports. This allows the write ports to "catch up" when writes are delayed due to contention. Writes are scheduled to perform bank access if they are within a write node. If there are no read ports that need to access the same bank, the writes can be dequeued from the write nodes. If the write ports cannot move the transactions to the write nodes because the write nodes are full, the writes to the write nodes are prioritized over reads.

[0029] Referring now to the drawings, FIG. 1 illustrates a computing environment 100 in one implementation. Computing environment 100 includes a CPU 101, a memory controller 103, and a memory block 109. CPU 101 is a representative processing resource (e.g., a processor core and / or a group thereof, with or without support circuitry) that needs to access memory to perform related functions. For example, CPU 101 can represent a system that uses a neural network to perform tasks (e.g., object detection, image classification, etc.).

[0030] CPU 101 generates read requests to be handled by memory controller 103. Memory controller 103 is coupled to memory block 109 to handle the read requests. In one implementation, CPU 101 includes a streaming engine that provides read requests to memory controller 103. Memory controller 103 includes command queues for each streaming engine of CPU 101. In operation, the streaming engines of CPU 101 provide read requests to the respective command queues of memory controller 103. In response, memory controller 103 examines the read requests in the command queues to determine which read requests to execute.

[0031] Memory controller 103 represents a controller configured to schedule and execute read requests based on the availability of the read requests. Memory controller 103 may be implemented in a larger context to function as an on-chip mechanism for processing the read requests described hereinafter with reference to FIG. 3. Memory controller 103 is coupled to memory block 109 and includes, but is not limited to, command queue 105 and command queue 107. Memory block 109 represents a memory element that includes a plurality of memory banks. Memory block 109 is shown to include a total of eight memory banks in one example, although more or fewer banks are possible.

[0032] In one implementation, the memory controller 103 includes circuit elements configured to specify priorities among command queues. The priority specification indicates to the memory controller 103 which command queue represents the priority command queue. More specifically, the priority specification indicates to the memory controller 103 which command queue to examine first during execution cycles. For example, if command queue 105 represents the priority command queue, the memory controller 103 examines command queue 105 first for available read requests before examining command queue 107, thereby giving priority to requests present within command queue 105.

[0033] In some examples, each memory bank of the memory block 109 represents an addressable single-port SRAM bank that accepts only one access every other clock cycle, which means that the SRAM bank of the memory block 109 enters a cool-down state after execution and before enabling the next execution. For example, if memory bank 7 is permitted access in the first clock cycle, memory bank 7 enters a cool-down state during the second clock cycle, so memory bank 7 remains unavailable until the third clock cycle.

[0034] In one implementation, the addresses of the SRAM banks are hashed to avoid offsets that would otherwise negatively impact the performance of the computing environment 100. For example, in the context of a neural network, it is common to have an access pattern with large strides that repeatedly return to the same subset of addresses when executing read requests. Due to every other access cycle of the SRAM banks employed, the performance of the strides where a subset of addresses are mapped to the same SRAM bank becomes very poor. In some cases, such access patterns result in a performance overhead of up to 300%. When the addresses are distributed among the SRAM banks (e.g., by hashing or other address assignment), such access patterns are avoided. A method for hashing the addresses of the memory banks is described in detail with respect to FIG. 7. In some cases, the addresses of the SRAM banks are statically hashed prior to live read / write operations.

[0035] FIG. 2 shows how a memory controller (i.e., memory controller 103) that includes multiple read queues operates, which is referred to herein as method 200. First, the method includes loading read requests into multiple read queues and designating one of the multiple read queues as the current priority read queue (step 201). The read requests can be generated by a processor associated with the memory controller (i.e., any of the CPUs 101). In one implementation, the associated processor includes a streaming engine configured to provide the read requests to respective read queues. For example, in the context of FIG. 1, the first streaming engine of CPU 101 provides read requests to command queue 105, and the second streaming engine provides read requests to command queue 107.

[0036] Next, for at least one read request among a plurality of read requests, the memory controller schedules the read request to be executed based on the availability of the read request (step 203). In some examples, for a read request to be considered available in the priority read queue, the associated memory bank must also be available. Alternatively, for a read request to be available in the non-priority read queue, the memory controller may require that the associated memory bank be available and that the read request in the non-priority read queue does not conflict with the most recently scheduled read request in the priority read queue. In operation, the memory controller sequentially scans the read requests in the priority read queue to determine a first read request available for execution. Next, the memory controller sequentially scans the read requests in the non-priority read queue to identify a second read request available for execution. Once the available read requests are identified from the priority and non-priority read queues, the memory controller schedules the requests available for execution. In one implementation, the memory controller executes the requests simultaneously. In another implementation, the memory controller executes the read requests in the priority read queue before executing the read requests in the non-priority read queue. In the context of FIG. 1, the memory controller 103 first scans the read requests in the priority read queue (e.g., command queue 105) to determine a first read request available for execution. The memory controller 103 then scans the read requests in the non-priority command queue (e.g., command queue 107) to determine a second read request available for execution. Once the available requests are identified from each queue, the memory controller 103 schedules and executes the requests.

[0037] If the read request scheduled and executed from the priority read queue is the first read request of the queue (i.e., the head read request), this method continues by designating different read queues of the plurality of read queues as the priority read queue (step 205). In one implementation, the memory controller includes a priority pointer configured to determine the priority designation among the plurality of read queues. The priority pointer reallocates the priority designation when the head read request of the priority read queue has been executed. In one implementation, the priority pointer reallocates the priority designation based on the capacity of the read requests in the remaining read queues. For example, the priority pointer may allocate priority to the read queue containing the most read requests. In another implementation, the priority pointer reallocates the priority designation to the read queue that has stored its head read request for the longest time. In the context of FIG. 1, the priority circuit element of the memory controller 103 allocates priority to the non-priority command queue (e.g., command queue 107) when the head read request of the priority command queue (e.g., command queue 105) has been executed. When the head read request of the new priority command queue has been executed, the priority circuit element returns to the priority designation.

[0038] Figure 3 illustrates a controller 300 in one implementation. The controller 300 includes, but is not limited to, a memory controller 319 (i.e., memory controller 103), a memory block 321 (i.e., memory block 109), and an arbitration network. The controller 300 can be coupled to one or more processors that send access requests to the controller 300 in support of software applications. Examples of software applications executed on the processor include, but are not limited to, artificial neural networks and other such machine learning algorithms (however, any type of software can be executed on the processor and can benefit from the memory technologies described herein). The controller 300 executes the requests and returns the data to the associated processor. The data is, for example, program code to be executed by the processor and / or actual data to be processed.

[0039] The arbitration network of the controller 300 is configured to receive requests from the associated processor and route the requests accordingly. The arbitration network of the controller 300 includes read routers 301 and 303, routers 305 and 307, read ports 311 and 315, and write ports 315 and 317. The routers of the arbitration network receive requests from the associated processor and route the requests to a specific read or write port accordingly. In one implementation, each router of the arbitration network responds to a requester of the associated processor. That is, the controller 300 includes a router for each requester of the associated processor. Since the processor does not require direct access to each bank of the memory block 321, the arbitration network reduces the number of global wires required by the controller 300. Instead, each requester of the associated processor provides a service request to the corresponding router to obtain access to the data in the memory block 321.

[0040] Read routers 301 and 303 represent components that manage only read requests. In one implementation, read routers 301 and 303 receive read requests from the respective streaming engines of the associated processors. For example, a first streaming engine may deliver a read request to read router 301, and a second streaming engine may deliver a read request to read router 303. In response, read router 301 routes the read request to read port 311, and read router 303 routes the read request to read port 313.

[0041] Routers 305 and 307 represent components that manage both read and write requests. In one implementation, routers 305 and 307 receive requests from the respective components of the associated processors. For example, a data memory controller (DMC) of the associated processor may deliver a request to router 305, and an extended level 2 cache (EL2) may deliver a request to router 307. In response, router 305 routes the request to either read port 311 or write port 315 depending on the type of the request. Similarly, router 307 routes the request to either read port 313 or write port 317.

[0042] Read ports 311 and 313 represent components configured to load read requests into the respective command queues of memory controller 319. In one implementation, memory controller 319 includes command queues for each read port of controller 300. In operation, read port 311 and read port 313 load read requests into the respective command queues of memory controller 319.

[0043] Write ports 315 and 317 represent components configured to load write requests into respective command queues of memory controller 319. In one implementation, memory controller 319 includes command queues for each write port of controller 300. In operation, write ports 315 and 317 load write requests into respective command queues of memory controller 319.

[0044] Memory controller 319 represents a controller configured to execute service requests. Memory controller 319 includes command queues for each read and write port of controller 300. The command queues of memory controller 319 store requests for specific routers. For example, a first command queue coupled to read port 311 stores read requests from read routers 301 and 305. Similarly, a second command queue coupled to read port 313 stores read requests from read routers 303 and 307. Alternatively, third and fourth command queues coupled to write ports 315 and 317 respectively store write requests from the corresponding routers. In one implementation, memory controller 319 represents the memory controller shown in FIGS. 1, 4A - 4C, 5A - 5B, and 6.

[0045] Memory block 321 is external to memory controller 319 and represents a memory element including a plurality of memory banks. In one implementation, each memory bank of memory block 321 represents an addressable single - port SRAM bank that accepts only one access every other clock cycle. Prior to operation, the addresses of the SRAM banks are hashed to avoid problematic offsets (described hereinafter with reference to FIG. 7). In one implementation, memory block 321 represents memory block 109 of FIG. 1.

[0046] In operation, the associated processor begins generating read and write requests for the router of controller 300. Read requests received by read router 301 or router 305 are sent to read port 311. Alternatively, read requests received by read router 303 or router 307 are sent to read port 313. Write requests received by router 305 are sent to write port 315, while write requests received by router 307 are sent to write port 317.

[0047] Upon receiving a request, each port of controller 300 provides the request to the corresponding command queue. In response, memory controller 319 begins executing the requests in the command queue. In one implementation, memory controller 319 processes read requests as shown in FIGS. 4A-4C using method 200. In another implementation, memory controller 319 processes write requests using an operation sequence as shown in FIGS. 5A-5B. If requested by the request, memory controller 319 returns the data of the request to the associated processor.

[0048] FIGS. 4A-4C illustrate an operation sequence for a memory controller configured to handle read requests, referred to herein as memory controller 401. Memory controller 401 can be implemented to process as an on-chip mechanism for handling read requests in a larger context. For example, memory controller 401 can represent memory controller 319 of controller 300. In such a context, memory controller 401 receives read requests from read port 311 and read port 313. Memory controller 401 is coupled to memory block 409 and includes, but is not limited to, priority pointer 403, command queue 405, and command queue 407. In other implementations, memory controller 401 includes more than two command queues, but for illustrative purposes, only two command queues are described herein.

[0049] The priority pointer 403 may represent hardware, software, firmware, or a combination thereof configured to allocate priority among the command queues of the memory controller 401. The priority designation assigned by the priority pointer 403 provides an indication to the memory controller 401 as to which command queue to service first. In operation, the priority pointer 403 reallocates the priority designation when the read request at the head of the priority command queue is executed. For example, the priority pointer 403 reallocates the priority from command queue 405 to command queue 407 when the read request at the head of command queue 405 is executed.

[0050] Command queues 405 and 407 represent queues configured to store read requests. Command queues 405 and 407 may indirectly receive read requests from a processor associated with the memory controller 401. For example, a first streaming engine of the associated processor may supply a read request for command queue 405, while a second streaming engine of the associated processor supplies a read request for command queue 407. In the context of FIG. 3, command queue 405 may receive a read request from read port 311, while command queue 407 receives a read request from read port 313. In one implementation, the memory controller 401 includes a command queue for each read requester (i.e., streaming engine) of the associated processor.

[0051] In one implementation, the read requests for command queues 405 and 407 identify the desired memory bank to execute the read request. For example, the currently scheduled read request for command queue 405 attempts an access to memory bank 0, while the currently scheduled read request for command queue 407 attempts an access to memory bank 7. Memory controller 401 determines which read request to schedule based on the availability and current prioritization of the associated memory bank.

[0052] Memory block 409 represents a memory element that includes a plurality of memory banks (e.g., memory block 109) arranged in parallel. In one implementation, each memory bank of memory block 409 represents an addressable single-port SRAM bank that accepts only one access every other clock cycle. That is, after execution, the SRAM banks of memory block 409 require a cooldown cycle to permit the next execution. Memory banks 1, 2, and 5 represent SRAM banks that were accessed in the previous cycle and thus require a cooldown period before enabling the next execution. Alternatively, the remaining banks of memory block 409 (i.e., memory banks 0, 3, 4, 6, and 7) are available. In one implementation, the addresses of the SRAM banks are hashed to avoid offsets that would be problematic for the performance of memory controller 401, which will be described hereinafter with reference to FIG. 7.

[0053] Referring now to FIG. 4A, stage 400A shows the first execution of a read request. First, the memory controller 401 identifies the current priority designation specified by the priority pointer 403. Next, the memory controller 401 starts scanning the priority command queue (i.e., command queue 405) to determine the first read request available for execution. That is, the memory controller 401 identifies the first read request in that queue having an available memory bank. Starting from the head of the command queue 405, the memory controller 401 checks the availability of memory bank 2. Since memory bank 2 is in the cool-down state, the memory controller 401 moves to the next read request in the command queue 405 and checks the availability of the corresponding memory bank. Since memory bank 0 is available for access, the memory controller 401 schedules the corresponding read request. Since the scheduled read request was not at the head of the queue, the command queue 405 maintains the priority designation.

[0054] When a read request for command queue 405 is scheduled, memory controller 401 starts scanning command queue 407 to determine the first read request available for execution from that queue. That is, memory controller 401 identifies a first read request that has an available memory bank and does not interfere with the most recently scheduled read request of the priority command queue. Starting from the head of command queue 407, memory controller 401 examines the availability of the head read request. Since the head read request conflicts with the most recently scheduled read request of command queue 405, memory controller 401 moves to the next read request in command queue 407 and examines the availability of the next read request. Since memory bank 2 is in a cool-down state, memory controller 401 moves to the next read request in command queue 407 and examines its availability. Memory bank 7 is available for access and the corresponding read request does not conflict with the most recently scheduled read request of command queue 405, so memory controller 401 schedules that read request. As a result, memory controller 401 executes the scheduled read request and returns the results to the respective requesters. In one implementation, memory controller 401 executes requests simultaneously. In another implementation, memory controller 401 executes the read requests of the priority read queue before executing the requests of the non-priority read queue.

[0055] Referring now to FIG. 4B, step 400B shows the next execution of memory controller 401 following the execution shown in step 400A. As a result of step 400A, command queues 405 and 407 receive new read requests to be stored. Also, memory block 409 indicates that memory banks 1, 2, and 5 are available and memory banks 0 and 7 are in a cool-down state.

[0056] When stage 400A is completed, memory controller 401 rechecks the priority designation of the command queue. Since the priority designation has not changed from that indicated in stage 400A, memory controller 401 starts scanning command queue 405 to determine the first read request in that queue that is available for execution. Starting from the head of command queue 405, memory controller 401 checks the availability of memory bank 2. Since memory bank 2 is currently available for access, memory controller 401 schedules the corresponding read request.

[0057] When scheduling the read request at the head of command queue 405, memory controller 401 starts scanning command queue 407 to determine the first read request in that queue that is available for execution. Starting from the head of command queue 407, memory controller 401 checks the availability of the head read request. Since memory bank 0 is in a cool-down state, memory controller 401 moves to the next read request in command queue 407 and checks the availability of the request. Since the next read request conflicts with the most recently scheduled read request in command queue 405, memory controller 401 moves to the next read request in command queue 407 and checks its availability. Since memory bank 3 is available for access and the corresponding read request does not conflict with the most recently scheduled read request in command queue 405, memory controller 401 schedules that read request. As a result, memory controller 401 executes the scheduled read requests and returns the results to their respective requesters. Also, since the read request at the head of command queue 405 has been executed, priority pointer 403 switches the priority designation to command queue 407.

[0058] Referring now to FIG. 4C, step 400C shows the next execution of memory controller 401 following the execution shown in step 400B. As a result of step 400B, command queues 405 and 407 receive new read requests, and memory block 409 indicates that memory banks 0 and 7 are available and memory banks 2 and 3 are in a cool-down state.

[0059] First, memory controller 401 identifies the current priority assignment as indicated by priority pointer 403. When step 400B is completed, priority pointer 403 transitions the priority assignment from command queue 405 to command queue 407. As a result, upon determining the first read request available for execution from that queue, memory controller 401 identifies the first read request of command queue 407 that has an available memory bank. Starting from the head of command queue 407, memory controller 401 checks the availability of memory bank 0. Since memory bank 0 is currently available for access, memory controller 401 schedules the corresponding read request.

[0060] After scheduling the read request at the head of command queue 407, memory controller 401 begins scanning command queue 405 to determine the first read request available for execution from command queue 405. Since command queue 405 does not hold a priority assignment, upon determining the first read request available for execution, memory controller 401 identifies the first read request of command queue 405 that has an available memory bank and does not interfere with the most recently scheduled read request of the priority command queue (i.e., command queue 407).

[0061] Starting from the head of the command queue 405, the memory controller 401 checks the availability of the head read request. Since memory bank 5 is available for access and the corresponding read request does not conflict with the most recently scheduled read request in command queue 407, the memory controller 401 schedules that read request. As a result, the memory controller 401 executes the scheduled read request and returns the results to the respective requesters. Also, since the head read request in command queue 407 is being executed, the priority pointer 403 switches the priority assignment and returns it to command queue 405. The priority assignment is changed only when the head read request of the priority command queue is executed. That is, the execution of the head read request from the non-priority command queue does not affect the priority assignment.

[0062] Figures 5A and 5B illustrate an operation sequence for a memory controller, configured to handle write requests and referred to herein as memory controller 501. The memory controller 501 may be implemented in a larger context to serve as an on-chip mechanism for handling write requests. For example, the memory controller 501 may represent the memory controller 319 of controller 300. In such a context, the memory controller 501 receives write requests from write ports 315 and 317. Although only two write ports (315 and 317) are shown, it can be understood that more than two write ports are possible. In one implementation, the memory controller 501 is further configured to handle read requests. For example, the memory controller 501 may represent memory controller 103 or memory controller 401, and thus elements for handling read requests are not shown.

[0063] The memory controller 501 includes, but is not limited to, the command queues 503, 505, and the write nodes 507, 511, 515, and 519. Only two command queues (503 and 505) are shown, but it can be understood that more than two command queues are possible. For example, the memory controller 501 may include a command queue for each write port of the associated device.

[0064] The command queues 503 and 505 represent queues configured to store write requests. In operation, the command queues 503 and 505 store write requests in the order in which the write requests are received. The command queues 503 and 505 may receive write requests indirectly from a processor associated with the memory controller 501. For example, the DMC of the associated processor may provide a write request to the command queue 503, while the EL2 of the associated processor provides a write request to the command queue 505. In one implementation, the command queues 503 and 505 receive write requests from respective ports (i.e., the write ports 315 and 317) of the associated device (controller 300). The command queues 503 and 505 provide the write requests to the write nodes 507, 511, 515, and 519. The write requests stored by the command queues 503 and 505 identify the desired write node to execute the request.

[0065] The write node 507 represents "write node 0" and is coupled to the memory banks 523 and 525. The write node 511 represents "write node 1" and is coupled to the memory banks 527 and 529. The write node 515 represents "write node 2" and is coupled to the memory banks 531 and 533. The write node 519 represents "write node 3" and is coupled to the memory banks 535 and 537.

[0066] In one implementation, each of the write nodes 507, 511, 515, and 519 includes a queue for each command queue of the memory controller 501. In operation, the command queues 503 and 505 provide write requests to the corresponding queues of the appropriate write nodes. The appropriate write node describes the write node coupled to the correct memory bank for performing the write request. For example, if command queue 503 stores a write request for memory bank 527, command queue 503 may provide the write request to queue 512 of write node 511. Alternatively, if command queue 505 stores a write request for memory bank 527, command queue 505 may provide the write request to queue 513 of write node 511. More specifically, command queue 503 may provide the write request to queue 508 of write node 507, queue 512 of write node 511, queue 516 of write node 515, or queue 520 of write node 519, depending on the requirements of the write request. Alternatively, command queue 505 may provide the write request to queue 509 of write node 507, queue 513 of write node 511, queue 517 of write node 515, or queue 521 of write node 519. There are more write nodes than command queues to allow multiple write transactions to be executed simultaneously from the same command queue.

[0067] In one implementation, the memory banks 523, 525, 527, 529, 531, 533, 535, and 537 represent addressable single-port SRAM banks that accept only one access every other clock cycle. That is, the memory banks 523, 525, 527, 529, 531, 533, 535, and 537 require a cool-down cycle before allowing the next execution. Before operation, the addresses of the memory banks are hashed to avoid the problematic offsets, which will be described later in FIG. 7.

[0068] Referring now to FIG. 5A, stage 500A shows the first execution of a write request. First, command queues 503 and 505 receive write requests from associated devices. Next, memory controller 501 dequeues the write requests at the head of command queues 503 and 505 into the corresponding queues of appropriate write nodes. In response, memory controller 501 determines the availability of the write requests stored by the queues of the write nodes. A write request can be executed only when the corresponding memory bank is available. A memory bank that is not available may be servicing different requesters or cooling down for the next execution.

[0069] In one implementation, memory controller 501 services both read requests and write requests. In some examples, by default, read requests take precedence over write requests. That is, when a read request and a write request request access to the same memory bank, memory controller 501 executes the read request before executing the write request. Thus, a write request may not be executed when the associated memory bank is servicing a read request. In one implementation, memory controller 501 gives precedence to write requests over read requests (directed to the same memory bank) when the queue of write nodes is full. That is, memory controller 501 executes the write requests of the full queue before executing any read requests that request access to the same memory bank as the write requests in the full queue. When the queue is no longer full, read requests are given precedence over write requests.

[0070] When examining the write nodes, the memory controller 501 makes the following determinations. First, the write requests in the queue 508 of the write node 507 are available for execution. Next, the write requests in the queue 513 of the write node 511 are not available for execution because the corresponding memory bank is currently servicing a different request (i.e., a read request). Finally, the write requests in the queue 516 of the write node 515 are not available for execution because the corresponding memory bank is in a cool-down state. As a result, the memory controller 501 executes the write requests in the queue 508. When the requests are executed, the command queues 503 and 505 receive new write requests from the associated devices, and the memory controller 501 dequeues the write requests at the heads of the command queues 503 and 505 into the corresponding queues of the appropriate nodes.

[0071] Here, referring to FIG. 5B, step 500B shows the next execution of the memory controller 501. When step 500A is completed, the memory controller 501 determines the availability of the write requests stored by the queues of the write nodes. When examining the write nodes, the memory controller 501 determines that the write requests in the queue 513 of the write node 511 are not available for execution because the corresponding memory bank is in a cool-down state. The memory controller 501 also determines that the write requests in the queues 516 and 517 of the write node 511 are available for execution.

[0072] In one implementation, to determine which write request to execute from the available options, the memory controller 501 examines the current capacity of the queues. For example, since the queue 516 contains more write requests than the queue 517, the memory controller 501 executes the commands from the queue 516. In another implementation, the memory controller 501 examines the urgency of the requests to determine which write request to execute from the available options.

[0073] When determining which request to execute, the memory controller 501 executes one of the available options. As a result, the command queues 503 and 505 receive new write requests, and the memory controller 501 provides the write requests at the head of the command queues 503 and 505 to the appropriate queues of the respective nodes. The operation sequences shown in steps 500A and 500B are repeated for the remaining write request executions.

[0074] FIG. 6 illustrates another architecture for the memory controller 501 of FIGS. 5A and 5B, which is referred to herein as architecture 600. Architecture 600 includes a write request pipeline network. The write request pipeline network of architecture 600 is inserted between the command queue and the write nodes of the memory controller 501 and is configured to route write requests from the command queue to the corresponding queues of the appropriate write nodes. The write request pipeline network of the memory controller 501 includes pipelines 603, 605, 607, and 609. In one implementation, the write request pipeline network includes a pipeline for each write node of the memory controller.

[0075] Pipeline 603 represents a network configured to load write requests from command queue 503 into the corresponding queues of write nodes 507 and 511. Pipeline 603 can load the write request into either queue 508 (not shown) of write node 507 or queue 512 (not shown) of write node 511, depending on the requirements of the write request. To determine to which write node to provide the request, pipeline 603 examines the memory bank requirements of the write request.

[0076] Pipeline 605 represents a network configured to load write requests from command queue 505 into the corresponding queues of write nodes 507 and 511. Pipeline 605 may provide a write request to either queue 509 (not shown) of write node 507 or queue 513 (not shown) of write node 511. When providing a request, pipeline 605 examines the memory bank requirements of the write request to determine which write node to provide the request to.

[0077] Pipeline 607 represents a network configured to load write requests from command queue 503 into the corresponding queues of write nodes 515 and 519. Pipeline 607 may provide a write request to either queue 516 (not shown) of write node 515 or queue 520 (not shown) of write node 519. Pipeline 607 examines the memory bank requirements of the write request to determine which write node to provide the request to.

[0078] Pipeline 609 represents a network configured to load write requests from command queue 505 into the corresponding queues of write nodes 515 and 519. Pipeline 607 may provide a write request to either queue 517 (not shown) of write node 515 or queue 521 (not shown) of write node 519. To determine which write node to provide the request to, pipeline 609 examines the memory bank requirements of the write request.

[0079] FIG. 7 illustrates a hashing process 700 for a memory controller (i.e., memory controller 103). The hashing process 700 represents a method of identifying the associated memory bank (i.e., the memory bank of memory block 109) that is the target of a read or write request.

[0080] Generally, a memory controller as contemplated herein may operate in one of two modes to identify a relevant memory bank. That is, in one mode, a portion of the address identified in a given request is directly used to assert the identification of the memory bank. In the other mode, a hashing algorithm is applied to the address to determine which bank the address maps to. The hashing process 700 is the latter, an algorithm that hashes the address identified in a read or write address to determine the associated memory bank.

[0081] When the advantages of the write process described herein are desired (e.g., to avoid systemic bank conflicts), an appropriate hashing algorithm is used on the write request. When the data to be retrieved has been written using a hashing algorithm, an appropriate hashing algorithm is used on the read request. In other words, if a write request that wrote data to its current location in memory was subject to a hashing algorithm, a read request looking for the same data may be subject to the same hashing algorithm. The following describes the steps performed by the memory controller when it is subject to the hashing process 700.

[0082] In operation, the memory controller receives a service request that identifies an address in memory. As described above, the availability of the memory bank associated with the service request affects whether the request is scheduled. To identify the relevant memory bank, the memory controller begins by multiplying the upper set of bits of the address by 5 (step 701), such that the upper set of bits of the address describes bits of the address that are larger than the bits that describe the byte offset within the bank. For example, if the memory bank is 16 bytes wide, each bit of the address greater than 4 is multiplied by 5. In one implementation, the upper set of bits goes through shift and addition operations to obtain the desired result.

[0083] Next, the memory controller divides the resulting address into a discrete number of bit fields (step 703). In one implementation, the number of bit fields is equal to the number of bits in the address after the multiplication operation divided by the binary logarithm of the number of memory banks in the system. For example, at the completion of step 701, in a system with 8 memory banks, if the address currently has 12 bits, the number of bit fields is equal to 12 divided by the binary logarithm of 8.

[0084] Upon dividing the address, the memory controller rotates the number of bit fields of the address based on the positions of the number of bit fields within the address (step 705). As a result, the address now contains a discrete number of rotated bit fields. Finally, the memory controller performs an exclusive NOR (XNOR) operation on the number of rotated bit fields within the address (step 707). As a result of the XNOR operation, the memory controller holds the identity of the memory bank associated with the service request and can proceed to determine its availability.

[0085] Alternative hashing processes are also possible. For example, a memory controller can XOR all address bits by dividing an address into N-bit fields and XORing those fields together to generate a final bank number (where N is the number of bits required to encode the bank number). For example, in the case of a 16-bank memory system with 16-byte wide banks, N = 4, and the function effectively XORs all nibbles of the address except the least significant nibble, which encodes the byte address within the bank. Using this technique, any sequence of addresses that stride by a power of two is guaranteed to proceed to different banks in subsequent cycles. For example, this can occur in a 4MB (22 address bits), 16-bank system with a stride of 0x400 (1KB). However, the drawback of such an alternative hashing function is that if there are strides with the same number in two of the N-bit fields, the generated bank addresses will periodically proceed to the same bank every 2^N cycles.

[0086] As will be understood by those skilled in the art, aspects of the present invention may be embodied as a system, method, or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware implementation, an entirely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, all of which may generally be referred to herein as a "circuit," "module," or "system." Further, aspects of the present invention may take the form of a computer program product embodied on one or more computer-readable media having computer-readable programs embodied thereon.

[0087] In fact, the description and figures included show specific implementations for teaching those skilled in the art how to generate and use the best mode. For the purpose of teaching the inventive principles, certain aspects have been simplified or omitted. Those skilled in the art will understand appropriate variations from these implementations that fall within the scope of the description. Those skilled in the art will also understand that the above features can be combined in various ways to form multiple implementations. As a result, the present invention is not limited to the specific implementations described above, but is limited only by the claims and their equivalents.

[0088] The foregoing description and the related figures teach the best mode of the present invention. The following claims define the scope of the present invention. It should be noted that some aspects of the best mode may not fall within the scope of the invention as specified by the claims. Those skilled in the art will understand that the above features can be combined in various ways to form multiple variations of the present invention. Thus, the present invention is not limited to the specific examples described above, but is limited only by the following claims and their equivalents.

Claims

1. It is a method, The storage of a set of read requests to a set of read queues, wherein the set of read queues includes a first read queue that is currently designated as a preferred read queue, With respect to the first read request of the set of read requests, a determination is made as to whether or not to schedule the first read request based on the availability of the associated memory bank, and also based on whether or not the first read request conflicts with a second read request from the preferred read queue if the first read request is in a queue of the set of read queues that is not the preferred read queue. Based on executing the second read request for the priority read queue, it is determined whether to designate a different read queue in the set of read queues as the priority read queue, based on whether the second read request was at the beginning of the priority read queue. Methods that include...

2. The method according to claim 1, A method further comprising determining that the associated memory bank is available and that the first read request is not in the priority read queue and does not conflict with the second read request, and then scheduling the first read request to be executed.

3. The method according to claim 1, A method further comprising determining that the associated memory bank is available and that the first read request is in the priority read queue, and then scheduling the first read request to be executed.

4. The method according to claim 1, A method for determining whether the associated memory bank is available, based on whether multiple cycles have occurred since the most recent access to the associated memory bank.

5. The method according to claim 1, A method in which the first read request conflicts with the second read request when the associated memory bank for the first read request is the same as the memory bank associated with the most recently scheduled read request.

6. The method according to claim 1, A method further comprising refraining from scheduling the first read request and moving on to the next read request in the queue, regardless of whether the queue is the priority read queue, if the associated memory bank is unavailable.

7. The method according to claim 1, A method that, when scheduling the first read request to be performed, further includes moving to the next read request in a different set of read queues, regardless of whether the queue is the preferred read queue.

8. The method according to claim 1, Loading write requests into multiple write queues, Processing the write request, which includes, for each write request, dequeuing the write request to a write node coupled to the associated memory bank, and scheduling the write request to be executed based on the availability of the associated memory bank; Methods that further include the above.

9. The method according to claim 1, Schedule the execution of the write request based on the availability of the associated memory bank, If it is determined that the associated memory bank is unavailable and the write node is not full, the write request will be withheld, and the next write request will be moved to a different write node. If it is determined that the associated memory bank is unavailable and the write node is full, the write request is given priority over a read request directed to the associated memory bank. When it is determined that the associated memory bank is available, the write request is scheduled to be executed, Methods that include...

10. It is a memory controller, A set of read queues, including the first read queue currently designated as the preferred read queue, A control circuit element coupled to the set of read queues, Load the read requests into the aforementioned set of read queues, For at least one of the read requests, the read request is scheduled to be executed based on the availability of the associated memory bank and, if the read request is in a queue other than the preferred read queue, whether the read request conflicts with a recently scheduled read request from the preferred read queue. When the recently scheduled read request of the priority read queue is executed, if the recently scheduled read request was at the beginning of the priority read queue, a different read queue in the set of read queues is designated as the priority read queue. The control circuit element is configured as follows: A memory controller, including the memory controller.

11. A memory controller according to claim 10, A memory controller further configured such that, upon determining that the associated memory bank is available, the read request is not in the priority read queue, and does not conflict with any recently scheduled read requests, the control circuit element schedules the read request to be executed.

12. A memory controller according to claim 10, A memory controller further configured such that, upon determining that the associated memory bank is available and that the read request is not in the priority read queue, the control circuit element schedules the read request to be executed.

13. A memory controller according to claim 10, A memory controller in which, for the associated memory bank to be available, multiple cycle requests must have occurred since the most recent access to the associated memory bank.

14. A memory controller according to claim 10, A memory controller in which the read request conflicts with the recently scheduled read request when the associated memory bank for the read request is the same as the memory bank associated with the recently scheduled read request.

15. A memory controller according to claim 10, A memory controller further configured such that, upon determining that the associated memory bank is unavailable, the control circuit element refrains from scheduling the read request and moves on to the next read request in the queue, regardless of whether the queue is the priority read queue.

16. A memory controller according to claim 10, A memory controller further configured such that, when the read request is scheduled to be executed, the control circuit element moves to the next read request in one of the different sets of read queues, regardless of whether the queue is the priority read queue or not.

17. A computing device, A set of memory banks, One or more processors coupled to the set of memory banks, A memory controller coupled to the set of memory banks and one or more processors, including a set of read queues, which includes a read queue currently designated as a priority read queue, A read request is loaded into the set of read queues from one or more of the aforementioned processors. For at least one of the read requests, the read request is scheduled to be executed based on the availability of the associated memory bank and, if the read request is in a queue other than the preferred read queue, whether the read request conflicts with a recently scheduled read request from the preferred read queue. When the most recently scheduled read request of the priority read queue is executed, if the most recently scheduled read request was at the beginning of the priority read queue, a different read queue in the set of read queues is designated as the priority read queue. The memory controller is configured as follows: A computing device, including a computer.

18. A computing device according to claim 17, The aforementioned memory controller When it is determined that the associated memory bank is available, the read request is not in the priority read queue, and does not conflict with the most recently scheduled read request, the read request is scheduled to be executed, and the process moves to the next request in the priority read queue. When it is determined that the associated memory bank is available and the read request is in the priority read queue, the read request is scheduled to be executed, and the system moves on to the next request in one of the different sets of read queues. If it is determined that the associated memory bank is unavailable, then, regardless of whether the queue is the priority read queue or not, the read request is not scheduled, and the process moves to the next read request in the queue. A computing device further configured in this way.

19. A computing device according to claim 17, A computing device in which, for the associated memory bank to be available, multiple cycles of requests must have occurred since the most recent access to the associated memory bank.

20. A computing device according to claim 17, A computing device in which the read request competes with the recently scheduled read request when the associated memory bank for the read request is the same as the memory bank associated with the recently scheduled read request.

Images