Scheduling memory requests with non-uniform latencies

Abstract

Disclosed are systems, devices, and methods for performing scheduling of memory requests to be transmitted to two different memory types. A computing system includes one or more clients for processing an application. A heterogeneous memory channel within a memory controller transmits memory traffic between the memory controller and a memory bus connected to each of a first memory and a second memory different from the first memory. The memory controller determines a next given point in time at which read response data has not been scheduled to be driven on the memory bus. The memory controller determines whether there is time to schedule a first memory access command for accessing the first memory and a second memory access command for accessing the second second memory. If there is enough time for each, one of the access commands is selected based on a weighted criterion.

Description

Background Art

[0001] Related technical description

[0002] A variety of computing devices utilize heterogeneous integration that integrates multiple types of ICs for providing system functionality. Multiple functions are provided in a processing node, and the multiple functions include audio / video (A / V) data processing, other high data parallel applications for medical and commercial fields, processing instructions of a general instruction set architecture (ISA), digital, analog, mixed signal, and radio frequency (RF) functions, etc. There are multiple options for placing a processing node in a system package to integrate multiple types of ICs. Some examples are system on chip (SOC), multi-chip module (MCM), and system-in-package (SiP).

[0003] Regardless of the choice of system packaging, in several uses, the performance of one or more computing systems may depend on processing nodes. In one example, a processing node is one of multiple processing nodes in a socket of a multi-socket server. The server is used to provide services to other computer programs in remote computing devices as well as computer programs in the server. In another example, a processing node is used within a mobile computing device that runs several different types of applications and may relay information to multiple users (both local and remote) at once.

[0004] Maintaining performance at a relatively high level generally requires fast access to stored data. Several types of data-intensive applications rely on fast access to data storage to provide reliable high performance for several local and remote programs and their users. The memory hierarchy changes from relatively fast volatile memory (such as registers on the processor die and caches on or connected to the processor die) to non-volatile and relatively slow memory. The interface and access mechanism for different types of memory also change. Therefore, any hybrid proposal for combining two different types of memory in a hierarchy brings challenges to maintaining high performance to meet the fast access requirements of running computer programs.

[0005] In view of the above, efficient methods and systems for scheduling memory requests to be issued to two different memory types are desired. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0007] Figure 1 is a block diagram of one embodiment of a computing system.

[0008] Figure 2 is a block diagram of one embodiment of a timing diagram.

[0009] Figure 3 is a block diagram of another embodiment of a timing diagram.

[0010] Figure 4 is a block diagram of another embodiment of a timing diagram.

[0011] Figure 5 is a block diagram of another embodiment of a timing diagram.

[0012] Figure 6 is a block diagram of another embodiment of a computing system.

[0013] Figure 7 is a block diagram of one implementation of a memory controller.

[0014] Figure 8 is a flow chart of one embodiment of a method for scheduling memory requests for transmission to two different memory types.

[0015] Fig. 9 is a flow chart of another embodiment of a method for scheduling memory requests for transmission to two different memory types.

[0016] Fig.10 is a flow chart of another embodiment of a method for scheduling memory requests for transmission to two different memory types.

[0017] Fig.11 is a flow chart of another embodiment of a method for scheduling memory requests for transmission to two different memory types.

[0018] Fig.12 is a flow chart of another embodiment of a method for scheduling memory requests for transmission to two different memory types.

[0019] Although the present invention is susceptible to various modifications and alternative forms, specific embodiments are shown in the drawings by way of example and are described in detail herein. However, it should be understood that the drawings and their detailed description are not intended to limit the invention to the specific forms disclosed, but on the contrary, the invention will cover all modifications, equivalents and alternatives falling within the scope of the invention as defined by the appended claims. DETAILED DESCRIPTION

[0020] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, it will be appreciated by those of ordinary skill in the art that various embodiments may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid confusion with the methods described herein. It should be understood that in order to make the description clear and simple, the elements shown in the drawings may not be drawn to scale. For example, the size of some elements may be enlarged relative to other elements. For example, the size of some elements may be enlarged relative to other elements.

[0021] Various systems, apparatus, methods, and computer-readable media for identifying response data that arrives out of order from two different memory types are disclosed. In various embodiments, a computing system includes one or more clients for processing an application. Examples of clients are general-purpose central processing units (CPUs), graphics processing units (GPUs), accelerated processing units (APUs), input / output (I / O) devices, etc. A heterogeneous memory channel within a memory controller transmits memory traffic between the memory controller and a memory bus connected to each of a first memory and a second memory.

[0022] In various embodiments, the first memory and the second memory utilize different data storage technologies and have different access latencies. For example, each of the first memory and the second memory may include one of a variety of random access memories (RAMs) such as a variety of dynamic random access memories (DRAMs), one of a variety of non-volatile (NV) dual in-line memory modules (DIMMs) such as NVDIMM-P, another type of data storage technology such as phase change memory (PCM), ferroelectric memory (FeRAM), magnetoresistive memory (MRAM), resistive memory (ReRAM or RRAM), three-dimensional (3D) cross-point (XPoint) memory, etc. Therefore, the difference between one or more access latencies of the first memory and one or more access latencies of the second memory may exceed a threshold. In some embodiments, the access latency of the first memory measured from the emission of a read command to the received response with valid data is on the scale of tens of nanoseconds. In various embodiments, the access latency of the second memory measured from the emission of a memory access command to the received response is on the scale of hundreds of nanoseconds. Therefore, the difference between the latencies exceeds hundreds of nanoseconds, which may be higher than a given threshold time amount.

[0023] In various embodiments, a command processor or other logic converts each received memory request into one or more commands. A scheduler in a memory controller determines whether there are two pending memory access commands, such as a first command for a first memory type and a second command for a second memory type. The scheduler determines whether each of the first command and the second command can be transmitted without causing a data conflict on a shared memory data bus. For example, in addition to the access latency of each of the first memory and the second memory, based on the time point for transmitting the selected command, the memory controller tracks the time point when the scheduling read response data arrives on the shared memory data bus. In some embodiments, the time point is measured in clock cycles. If selecting any of the first command and the second command will not schedule a data conflict on the shared memory data bus, each of the first command and the second command is still a candidate for transmission. In this case, the scheduler selects a command from the first command and the second command based on arbitration logic.

[0024] In other embodiments, to avoid data conflicts on the shared memory data bus, a scheduler in the memory controller determines the next given time point at which read response data has not yet been scheduled to be driven on the memory data bus. The scheduler determines whether there is time to schedule a first memory access command for accessing the first memory to provide response data at the given time point. The scheduler also determines whether there is time to schedule a second memory access command for accessing the second memory to provide response data at the given time point.

[0025] If there is enough time for at least one of the first access command and the second access command to provide response data at a given point in time, the scheduler selects one of the first memory access command and the second memory access command based on arbitration logic. In one embodiment, the arbitration logic uses weighted criteria. The criteria include at least priority level, age, etc. Thereafter, the scheduler transmits the selected access command to one of the first memory and the second memory via the heterogeneous memory channel.

[0026] In some embodiments, when the scheduler schedules a given command to be transmitted, the scheduler determines the given time point at which the requested read data will be scheduled to arrive on the shared memory data bus. In one embodiment, the scheduler adds the waiting time of the given command to the time point at which the scheduler schedules to transmit the given command. In some embodiments, the scheduler generates an identifier as an indication of an entry in the request queue storing the information corresponding to the given command. In other embodiments, the identifier is generated based on a combination of one or more of a thread identifier and a part of the target address of the memory request corresponding to the given command. The scheduler stores the association of the identifier with the given time point. In one embodiment, a table is used. Therefore, the scheduler can identify the given command based on the requested read data on the shared memory data bus at a given time point instead of based on the tag inserted in the given command or by the grouping associated with the requested read data arriving.

[0027] refer to Figure 1 , a generalized block diagram of one embodiment of a computing system 100 is shown. As shown, clients 110 and 112 send memory requests to a memory controller 120. A heterogeneous memory channel 124 within the memory controller 120 transmits memory traffic between the memory controller 120 and the memory bus 130. Each of the memory 140 and the memory 150 stores data accessed by the clients 110 and 112. In various embodiments, the memory 140 and the memory 150 use different data storage technologies and therefore have different access latencies. In various embodiments, one or more of the memory 140 and the memory 150 are used as system memory by the clients 110 and 112.

[0028] For ease of illustration, communication fabrics, I / O interfaces for input / output (I / O) devices, and any links and interfaces for network connections are not shown in computing system 100. In some embodiments, components of computing system 100 are separate dies on an integrated circuit (IC), such as a system on a chip (SOC). In other embodiments, components are separate dies in a system-in-package (SiP) or a multi-chip module (MCM). In some embodiments, clients 110 and 112 include one or more of a central processing unit (CPU), a graphics processing unit (GPU), a hub for a multimedia engine, and the like. Each of clients 110 and 112 is one of a variety of computing resources capable of processing applications and generating memory requests.

[0029] Although a single memory controller 120 is shown, in other embodiments, another number of memory controllers are used in the computing system 100. In various embodiments, the memory controller 120 receives memory requests from the clients 110 and 112, and the scheduler 122 schedules the memory requests and sends the scheduled memory requests to one of the memories 140 and 150 via the heterogeneous memory channel 124. In some embodiments, the scheduler 122 within the memory controller 120 includes control logic that schedules memory requests that target memory locations in the memory 140 separately from scheduling memory requests that target memory locations in the memory 150. Thereafter, the scheduler 122 selects between memory requests that target the memory 140 and memory requests that target the memory 150. In one embodiment, the scheduler 122 intermixes accesses that target the memory 140 and the memory 150.

[0030] The control logic in the scheduler 122 for scheduling memory requests uses information such as the quality of service (QoS) or other priority level of the memory request, the process or software thread identifier (ID) of the memory request, the age of the memory request, the amount of time since the memory request was issued to the memory 140, the amount of time since the memory request was issued to the memory 150, etc. Thus, the scheduler 122 supports out-of-order issuance of memory requests. When the scheduler 122 selects a memory request to send to one of the memory 140 and the memory 150, the scheduler 122 sends the selected memory request to the heterogeneous memory channel 120 for transmission.

[0031] The heterogeneous memory channel 124 interfaces with each of the memory 140 and the memory 150. The heterogeneous memory channel 124 supports a protocol for interfacing with the memory 140 and supports another protocol for interfacing with the memory 150. The protocols determine values ​​for information transmission, such as the number of data transfers per clock cycle, signal voltage levels, signal timing, signal and clock phases, and clock frequencies.

[0032] In various embodiments, memory bus 130 supports sending data traffic in a single direction for a given amount of time (such as during a given mode of read and write modes) and then sending data traffic in the opposite direction for another given amount of time (such as during another mode of read and write modes). In one embodiment, memory bus 130 utilizes a single command bus and a single data bus. Thus, scheduling the transmission of memory requests to memory 140 and memory 150 is performed in a manner that avoids data conflicts on memory bus 130.

[0033] As described earlier, memory 140 and memory 150 use different data storage technologies and therefore have different access latencies. As shown, memory 140 has an access latency 132 that differs from an access latency 134 of memory 150 by at least a threshold amount of time. Although a single access latency is shown for each of memory 140 and memory 150, in other embodiments, one or more of memory 140 and memory 150 has multiple access latencies. However, each of the multiple access latencies of memory 140 differs from each of the multiple access latencies of memory 150 by at least the threshold amount of time.

[0034] In one embodiment, one of the memory 140 and the memory 150 includes one of a variety of dynamic random access memories (DRAMs), and the other of the memory 140 and the memory 150 includes one of a variety of non-volatile (NV) dual in-line memory modules (DIMMs), such as NVDIMM-P. In other embodiments, other memory types with different access latencies are used for the memory 140 and the memory 150. For example, in addition to using multiple types of random access memory (RAM) technology and NVDIMM technology, in some embodiments, each of the memory 140 and the memory 150 includes other examples of data storage technology, such as phase change memory (PCM), ferroelectric memory (FeRAM), magnetoresistive memory (MRAM), resistive memory (ReRAM or RRAM), three-dimensional (3D) cross-point (XPoint) memory, etc. In various embodiments, the difference between the access latency of the memory 140 and the access latency of the memory 150 is higher than a threshold. Therefore, the scheduler 122 includes control logic and sequential elements for transmitting memory access commands targeting locations in the memory 140 and the memory 150 in a hybrid manner.

[0035] In some embodiments, the memory controller 120 includes a command processor for converting each received memory request into one or more commands. In one embodiment, the scheduler 122 determines whether there are two pending memory access commands, such as a first command for memory 140 and a second command for memory 150. The scheduler 122 determines whether each of the first command and the second command can be transmitted without causing data conflicts on the shared memory data bus 130. For example, based on the time point for transmitting the selected command, the access latency 132 and the access latency 134, the memory controller 120 tracks the time point when the data will be scheduled to arrive on the shared memory data bus 130. The pending first command and the second command can be a read access or a write access. In some embodiments, the time point is measured in clock cycles. If the selection of any of the first command and the second command does not schedule the data conflict on the shared memory data bus 130, each of the first command and the second command is still a candidate for transmission. In this case, the scheduler 122 selects the command from the first command and the second command based on the arbitration logic. In one embodiment, the arbitration logic uses a weighted criterion.

[0036] In other embodiments, in order to avoid data conflicts on the memory bus 130 regardless of the obvious difference between the access latency 132 and the access latency 134, however, the scheduler 122 determines that the memory bus 130 is scheduled to be available at the next given time point. In other words, the scheduler 122 determines that the read response data or write data has not yet been scheduled to drive the next given time point on the memory bus 130. In some embodiments, the time point is measured by clock cycles. The scheduler 122 also determines whether there is time to schedule the first command for accessing the memory 140 and the second command for accessing the memory 150 to provide data at a given time point. As described earlier, the command processor converts the received memory request into a command. In one embodiment, one or more of the first command and the second command have one or more preceding commands and / or one or more following commands, and when the first command and the second command can be transmitted, the preceding command and / or the following command increase the waiting time and delay.

[0037] If there is enough time for at least one of the first access command and the second access command to provide data at a given point in time when the memory bus 130 is available, the scheduler 122 selects one of the first memory access command and the second memory access command. The scheduler 122 may use the criteria described earlier, such as priority level, age, etc. Thereafter, the scheduler 122 sends the selected access command to one of the memory 140 and the memory 150 via the heterogeneous memory channel 124.

[0038] refer to Figure 2, a generalized block diagram of one embodiment of a timing diagram 200 is shown. In the illustrated embodiment, memory access commands are shown as being issued at different times on the timeline. The memory access commands are issued to one of two different types of memory having different access latencies. In various embodiments, a first type of memory (memory type 1) uses a different data storage technology than a second type of memory (memory type 2), and therefore, memory type 1 has an access latency that differs from the access latency of memory type 2 by at least a threshold amount of time.

[0039] As shown, three memory access commands, labeled A, B, and C, are emitted at times indicated by markers t1, t2, and t3. These memory access commands are emitted to memory type 1. Responses to these memory access commands are shown as arriving at times indicated by markers t4, t5, and t6. In some embodiments, the markers on the timeline are equivalent to clock cycles. In other embodiments, the markers on the timeline are equivalent to other time measurements indicating a given point in time. In addition to having deterministic access wait times for the three markers on the timeline, responses are also shown as arriving in order relative to the order in which the memory access commands A, B, and C are emitted.

[0040] Additionally, another memory access command D is shown as being transmitted at a time indicated by marker t7. Memory access command D is transmitted to memory type 2. A response is shown as being received at a time indicated by marker t12. The access latency of memory access command D transmitted to memory type 2 is greater than the access latency of memory access commands A, B, and C transmitted to memory type 1. In some embodiments, the access latency of memory type 2 is five markers on the timeline.

[0041] In the illustrated embodiment, memory type 2 has a second access latency. For example, memory access command E is transmitted to memory type 2 at a time indicated by mark t13 on the timeline. In some embodiments, the second access latency of memory type 2 is six marks on the timeline. As shown, the response to memory access command E is shown as arriving at a time indicated by mark t19. In some embodiments, for memory type 1 and memory type 2, the read access latency is equal to the write access latency. In other embodiments, for one or more of memory type 1 and memory type 2, the read access latency is different from the write access latency. In the illustrated embodiment, access commands AE have different access latencies 210 and 220, and commands AE are shown as being transmitted separately from each other. However, this transmission scheme is inefficient.

[0042] refer to Figure 3, a generalized block diagram of another embodiment of a timing diagram 300 is shown. In the illustrated embodiment, memory access commands are shown as being issued at different times on a timeline. The memory access commands are issued to one of two different types of memory having significantly different access latencies (such as differing by at least a threshold amount of time). As shown, the latency 310 of access commands issued to memory type 1 is less than the latency 320 of access commands issued to memory type 2.

[0043] Similar to timing diagram 200, in some embodiments, the marks on the timeline are equivalent to clock cycles. In other embodiments, the marks on the timeline are equivalent to other time measurements indicating time points. As shown, three memory access commands labeled A, B, and C are emitted at times indicated by marks t1, t2, and t3. These memory access commands are emitted to memory type 1. In addition to the deterministic waiting time with the three marks on the timeline, the responses to these memory access commands are shown as arriving at times indicated by marks t4, t5, and t6, and are orderly relative to the order in which the memory access commands A, B, and C were emitted.

[0044] A memory access command D is transmitted to memory type 2 at a time indicated by marker t7. Before a response is received, another memory access command E is transmitted to memory type 1 at a time indicated by marker t8. At a time indicated by marker t9, it is not possible to transmit another memory access command to memory type 1 without a data conflict occurring. In this example, it is known that the access latency of the memory access command transmitted to memory type 1 is three marks on the timeline, and the access latency of the memory access command transmitted to memory type 2 is at least five marks on the timeline. Therefore, it is known that the memory data bus is not available at a time indicated by marker t12 due to the scheduling of the memory access command D. If a memory access command is transmitted to memory type 1 at a time indicated by marker t9, a data conflict will occur at t12.

[0045] Other access commands are shown being issued, such as memory access command F issued to memory type 2 at t13 and memory access commands G and H issued to memory type 1 at times t14 and t15. In this example, the access latency of memory access command F is six marks on the timeline. Therefore, it is known that the memory data bus is unavailable at the time indicated by mark t19 due to the scheduling of memory access command F. If the memory access command is issued to memory type 1 at the time indicated by mark t16, a data conflict will occur at t19. Therefore, the scheduler that issues memory access commands to both types of memory via the heterogeneous memory channel takes into account the time that the memory data bus is unavailable due to the latency of the commands in order to avoid data conflicts on the memory data bus.

[0046] refer to Figure 4 , a generalized block diagram of another embodiment of a timing diagram 400 is shown. In the illustrated embodiment, memory access commands 420 and 430 are shown as being transmitted at different times based on clock 410. In the illustrated embodiment, the clock cycle of clock 410 is used to provide a time measurement to identify a point in time. The memory access commands are transmitted to one of two different types of memory having access latencies that differ from each other by at least a threshold amount of time. In one embodiment, command 420 is transmitted to a first type of memory (memory type 1), and command 430 is transmitted to a second type of memory (memory type 2).

[0047] As shown, the waiting time of access command 420 is less than the waiting time of access command 430. For ease of illustration, the waiting time is not drawn to scale. In some embodiments, the access waiting time of memory type 1 measured from the emission of the read command to the received response with valid data is on the scale of tens of nanoseconds. In the illustrated example, the waiting time is shown as 2 clock cycles. In various embodiments, the access waiting time of memory type 2 measured from the emission of the memory access command to the received response (which may or may not include valid data) is on the scale of hundreds of nanoseconds. For ease of illustration, the waiting time is shown as 5 clock cycles, rather than drawn to scale.

[0048] In various embodiments, a memory request (such as a memory read request or a memory write request) is converted into one or more commands based on accessing the memory. For example, the control logic within the DRAM executes a complex transaction (such as an activation (open) transaction and precharge of data and control lines within the DRAM) once to access the identified row, and once during a close transaction to put the modified content stored in the row buffer back to the identified row. Each of the different DRAM transactions (such as activation / open, column access, read access, write access, and precharge / close) has a different corresponding latency. Typically, activation and precharge transactions have significantly higher latency than read access and write access transactions.

[0049] The dotted lines of the commands shown in the illustrated example represent possible additional commands that are transmitted together with the memory access command. For example, an activation / open command and a precharge / close command for DRAM may be used, but the command is not shown in the timing diagram 400. Similarly, for NVDIMM-P, each of the transaction read (X-READ) command, the send read (SREAD) command, and the speculative state read command is usually followed by an extended address (AXDR) command, which allows the address to be extended for large linear addresses. These additional commands are not specifically shown, but are represented by the dotted lines of the possible arrangement between commands 420 and 430. Therefore, back-to-back access commands are usually not transmitted on back-to-back clock cycles. The scheduler of the memory controller with heterogeneous memory channels considers possible additional commands when scheduling memory access commands for transmission.

[0050] Responses are shown as responses 440, and they are received on a shared single memory data bus. As shown, a memory access command "READ A" for memory type 1 is transmitted at clock cycle (CC) 1. With an access latency of two clock cycles in this example, valid response data arrives at CC3. As shown in this example, valid data consumes two clock cycles, such as CC 3 and CC 4. During each clock cycle, an amount of data equal to the width of the data bus is returned to the memory controller. The supported size of the data bus is based on design choice.

[0051] In one embodiment, the scheduler or other control logic in the memory controller determines that the next given point in time at which the memory data bus is scheduled to be available is after CC 4, i.e., CC 5. The scheduler determines that there is time to schedule the memory access command for memory type 1 and the memory access command for memory type 2. The amount of response data for the memory access command for memory type 1 will not conflict with the arrival of the response data of the earlier memory access command for memory type 2. Therefore, the scheduler transmits a read command "READ B" for memory type 2 at CC 2 and transmits a memory access command "READ C" for memory type 1 at CC 3. In the case of an access latency of two clock cycles in this example, valid response data for "READ C" arrives at CC 5 and CC 6. In the case of an access latency of five clock cycles in this example, for "READ B", valid response data is scheduled to arrive at CC 7 and CC 8. However, as shown in the figure, the requested data is not yet ready to be retrieved from memory type 2. An indication indicating that the requested data is not yet available is received by the memory controller and used by the scheduler to retry at a later time.

[0052] The scheduler determines that the next given point in time at which read response data has not been scheduled to be driven on the memory data bus is CC 9. The scheduler determines that there is time to schedule a memory access command for memory type 1 and a memory access command for memory type 2. To select the next memory access command to transmit, the scheduler uses information such as the quality of service (QoS) or other priority level of the memory request, the process or software thread identifier (ID) of the memory request, the age of the memory request, the amount of time since the memory access request was transmitted to memory type 1, the amount of time since the memory access request was transmitted to memory type 2, etc. In the example provided, the scheduler transmits a read access command "READ D" for memory type 2 at CC 4. With an access latency of five clock cycles in this example, valid response data is scheduled to arrive at CC 9 and CC 10.

[0053] The scheduler determines that the next given point in time when the memory data bus is available is CC 11. The scheduler determines that there is time to schedule a memory access request for memory type 1 and a memory access request for memory type 2. The scheduler selects the next memory access command to transmit based on earlier criteria such as priority level, age, etc. In some embodiments, the scheduler assigns a given weight to each of the criteria and performs a weighted sum. The memory access command with the largest sum is selected for transmission.

[0054] In one embodiment, the memory controller receives an indication on another channel or link interface that response data for "READ B" is now available from memory type 2. Although the memory access command "READ E" has a higher weighted sum than the send read command "SREAD B" corresponding to the earlier read command "READ B", the scheduler determines that the amount of response data for the memory access command "READ E" will conflict with the arrival of response data for the earlier read command "READ D". Therefore, the scheduler transmits the send read command "SREAD B" at CC 8 and transmits the memory access command "READ E" at CC 9. With an access latency of two clock cycles in this example, valid response data for "READ E" arrives at CC11 and CC 12. With an access latency of five clock cycles for "SREAD B" in this example, valid response data is scheduled to arrive at CC 13 and CC 14 (not shown). Although timing diagram 400 is described with respect to a read access command, in other embodiments, a similar timing diagram is used for a write access command, where write data is placed on a shared memory data bus and may have data conflicts with read response data or other write data of other write access commands.

[0055] In some embodiments, the received response data includes a tag or other identifier that identifies which command is associated with the response data. In other embodiments, the arrival timing of the request data is used to identify which command is associated with the response data. Therefore, although the request data arrives out of the order corresponding to the transmission of the command, the scheduler in the memory controller can track which received data belongs to which command.

[0056] refer to Figure 5 , a generalized block diagram of another embodiment of a timing diagram 500 is shown. In the illustrated embodiment, memory access commands 520 and 530 are shown as being emitted at different times based on clock 510. In the illustrated embodiment, the clock cycle of clock 510 is used to provide a time measurement to identify a point in time. Memory access commands are emitted to one of two different types of memory having different access latencies. In one embodiment, command 520 is emitted to a first type of memory, which is a conventional DRAM, and command 530 is emitted to a second type of memory, which is an NVDIMM-P. However, other types of memory having different access latencies are possible and contemplated.

[0057] For ease of illustration, command latency is not drawn to scale. In some embodiments, the command latency of conventional DRAM is on the scale of tens of nanoseconds. In the illustrated example, the latency is shown as 2 clock cycles. In various embodiments, the access latency of NVDIMM-P is on the scale of hundreds of nanoseconds. In the illustrated example, the latency is shown as 7 clock cycles. In various embodiments, a memory request (such as a memory read request) is converted into one or more commands based on accessing the memory. As described earlier, the control logic within the DRAM performs complex transactions, such as activating transactions and closing transactions. In addition, other signals are generated, such as the strobe 560 of the row address and the column address.

[0058] Similar to the earlier timing diagram 400, the timing diagram 500 is described with respect to a read access command. However, in other embodiments, a similar timing diagram is used for a write access command, wherein the write data is set on a shared memory data bus and may conflict with the read response data or other write data of other write access commands. Responses are shown as responses 540, and they are received on a single memory data bus. The scheduler selects the next memory access command to transmit based on earlier criteria such as priority, age, etc. In some embodiments, the scheduler assigns a given weight to each of the criteria and performs a weighted sum for use in selecting the next command to transmit.

[0059] As shown, the scheduler transmits a transaction read command "X-READ A" for memory type 2 at CC 2. An extended address command "XADR A" that allows extended addresses for large linear addresses follows at CC 3. With an access latency of 7 clock cycles in this example, valid response data is scheduled to arrive at CC 9. In some embodiments, the latency is measured from the command "XADR A" rather than from the command "X-READ A". In various embodiments, the requested data consumes multiple clock cycles. However, for ease of illustration, the requested data of the command "X-READ A" consumes a single clock cycle.

[0060] The scheduler transmits a memory access command "READ B" for memory type 1 at CC 3. With an access latency of two clock cycles in this example, valid response data arrives at CC 5. As shown, an activation command "ACTIVATE" is transmitted at CC 1 to prepare for transmitting a command "READ B" at CC 3. The column address strobe (CAS) on strobe 560 is asserted at CC 3 with a logic low value. The row address and column address are provided on the address line marked as pointer 570, aligned with the assertion of the corresponding strobe. As shown, the requested data of the command "READ B" consumes four clock cycles, such as CC 5, CC 6, CC 7 and CC 8. The scheduler takes into account the number of clock cycles consumed by the received requested data when determining the next given time point at which the memory data bus is available.

[0061] In one embodiment, the scheduler determines that the next given point in time that the memory data bus is available is CC 10. The scheduler determines that there is time to schedule a memory access command for memory type 1, but there is no time to schedule a memory access command for memory type 2. As shown in the figure, the earliest point in time to transmit the next memory access command for memory type 2 is after the command "XADR A", that is, CC 4. With a command latency of 7 clock cycles, the requested data is scheduled to arrive at CC 11 instead of CC 10. Therefore, the scheduler transmits a memory access command "READ C" for memory type 1 at CC 8. With an access latency of two clock cycles in this example, valid response data arrives at CC 10.

[0062] As shown, the precharge command "PRECHARGE" and the activation command "ACTIVATE" are transmitted at CC 4 and CC 6, respectively, to prepare for the transmission of the command "READ C" at CC 8. The storage bank to be closed is indicated in the "BANK" data on the address line marked as pointer 570. In some embodiments, the received response data includes a tag or other identifier that identifies which command is associated with the response data. In other embodiments, the arrival timing of the request data is used to identify which command is associated with the response data. Therefore, although the request data arrives out of the order corresponding to the transmission of the command, the scheduler in the memory controller can track which received data belongs to which command.

[0063] refer to Figure 6, a generalized block diagram of another embodiment of a computing system 600 is shown. As shown, the computing system 600 includes a communication fabric 620 between each of the clients 610 and a memory controller 630. The memory controller 630 includes a heterogeneous memory channel 638 for transmitting memory traffic between the memory controller 630 and the memory 670 and the memory 680 via a memory bus 650. Each of the memory 670 and the memory 680 stores data accessed by the client 610. In some embodiments, the components of the system 600 are separate dies on an integrated circuit (IC), such as a system on a chip (SOC). In other embodiments, the components are separate dies in a system-level package (SiP) or a multi-chip module (MCM). For ease of illustration, power controllers, interrupt controllers, network link interfaces, etc. are not shown.

[0064] In various embodiments, memory bus 650 utilizes a bidirectional shared bus structure. In various embodiments, memory 670 and memory 680 use different data storage technologies, and therefore, memory 670 has an access latency that differs from an access latency of memory 680 by at least a threshold amount of time. In various embodiments, one or more of memory 670 and memory 680 are used by client 610 as system memory.

[0065] In one embodiment, when one of the memory 670 and the memory 680 is one of multiple types of DRAM, an example of a protocol for the corresponding interface between the heterogeneous memory channel 638 and the memory controller 630 is a double data rate (DDR) type protocol. The protocol determines values ​​for information transfer, such as the number of data transfers per clock cycle, signal voltage levels, signal timing, signal and clock phases, and clock frequency. Protocol examples include DDR2 SDRAM, DDR3 SDRAM, GDDR4 (Graphics Double Data Rate, Version 4) SDRAM, GDDR5 SDRAM, GDDR6, HBM2, etc. The memory controller 630 includes control circuits for interfacing to the heterogeneous memory channel 638 and other memory channels (not shown) and following the corresponding protocol.

[0066] Although a single memory controller 630 is shown, in other embodiments, another number of memory controllers are used in the computing system 600. As shown, the memory controller 630 includes a request queue 632 for queuing memory access requests received from the client 610 via the communication fabric 620. The memory controller 630 also has a response queue 634 for storing responses received from the memory 670 and the memory 680. In one embodiment, the request queue 632 includes a separate read queue for each of the memory 670 and the memory 680 for storing memory read requests. In addition, the request queue 632 includes a separate write queue for each of the memory 670 and the memory 680 for storing memory write requests. In some embodiments, when one or more of the memory 670 and the memory 680 includes a data storage technology that provides a miss status as a response to an access, the memory controller 630 also includes a miss queue 639. In one embodiment, one of the memory 670 and the memory 680 is an NVDIMM-P that provides a miss status response.

[0067] In some embodiments, the request queue 632 includes one or more queues for storing received memory access requests, and a separate queue for storing scheduled memory access commands converted from the received requests and selected from the one or more queues. The scheduler 636 includes control logic for selecting memory access commands stored in the request queue 632 for out-of-order transmission to the memory 670 and the memory 680. Thus, the memory controller 630 supports out-of-order transmission of memory access commands to the memory 670 and the memory 680.

[0068] In various embodiments, a scheduler 636 in the memory controller 130 schedules the transmission of stored memory access commands based on quality of service (QoS) or other priority information, age, process or thread identifier (ID), amount of time since the memory access command was issued to memory 670, amount of time since the memory access command was issued to memory 680, and relationship to other stored requests (such as targeting the same memory channel, targeting the same rank, targeting the same memory bank, and / or targeting the same page). In some embodiments, the scheduler 636 assigns a given weight to each of the criteria and performs a weighted sum. The memory access command or state access command with the largest sum is selected for transmission.

[0069] In various embodiments, the communication fabric 620 transmits traffic back and forth between the client 610 and the memory controller 630 and includes interfaces for supporting corresponding communication protocols. In some embodiments, the communication fabric 620 includes at least queues for storing requests and responses, selection logic for arbitrating between received requests before sending them across the internal network, logic for building and decoding packets, and logic for routing packets.

[0070] In the illustrated embodiment, client 610 includes a central processing unit (CPU) 612, a graphics processing unit (GPU) 614, and a hub 616. Hub 616 is used to communicate with a multimedia engine 618. CPU 612, GPU 614, and multimedia engine 618 are examples of computing resources capable of processing application programs. Although not shown, in other embodiments, other types of computing resources are included in client 610. In some embodiments, each of the one or more processor cores in CPU 612 includes a circuit for executing instructions according to a given selected instruction set architecture (ISA). In various embodiments, each of the processor cores in CPU 612 includes a superscalar multithreaded microarchitecture for processing instructions of a given ISA.

[0071] In one embodiment, GPU 614 includes a high parallel data micro-architecture with a large number of parallel execution channels. In one embodiment, the micro-architecture uses a single instruction multiple data (SIMD) pipeline for parallel execution channels. Multimedia engine 618 includes a processor for processing audio data and visual data of multimedia applications. In some embodiments, the address space of computing system 600 is divided between at least CPU 612, GPU 614 and hub 616 and one or more other components (such as input / output (I / O) peripherals (not shown)) and other types of computing resources. Maintain memory mapping for determining which addresses are mapped to which component, and therefore determine which one of CPU 612, GPU 614 and hub 616 should be routed to a memory request for a specific address.

[0072] In various embodiments, one or more of memory 670 and memory 680 is filled with data from disk storage 662 through I / O controller and bus 660 and memory bus 650. The corresponding cache fill line with the requested block is delivered from one or more of memory 670 and memory 680 to a corresponding one of the cache memory subsystems in client 610 to complete the original memory access request. The cache fill line is placed in one or more levels of cache. In one embodiment, disk storage 662 provides non-volatile secondary storage for data. In one embodiment, disk storage 662 includes one or more hard disk drives (HDD). In other embodiments, disk storage 662 includes a solid state disk (SSD).

[0073] refer to Figure 7 , a generalized block diagram of one embodiment of a memory controller 700 is shown. In the illustrated embodiment, the memory controller 700 includes: an interface 710 to a client via a communication fabric; a queue 720 for storing received memory access requests and received responses; a control unit 750; and an interface 780 to multiple memory devices (each of which uses a different memory technology) via a memory data bus and a heterogeneous memory channel. Each of the interfaces 710, 780, and 782 supports a corresponding communication protocol. In one embodiment, the interface 780 is an interface to a memory command bus for sending a memory access command corresponding to a memory request received via the interface 710 to a memory device, the memory device including a data storage technology of a first memory type. In one embodiment, the interface 782 is an interface to a memory data bus for transmitting data between the memory controller 700 and another memory device, the other memory device including a data storage technology of a second memory type different from the first memory type. In various embodiments, the access latency of the first memory type differs from the access latency of the second memory type by at least a threshold amount of time.

[0074] In the illustrated embodiment, the queue 720 includes a request queue 730, a response queue 740, and a miss queue 742. In one embodiment, the queue 720 includes a first read queue 732 for storing received read requests targeted at the first memory type, and a second read queue 734 for storing received read requests targeted at the second memory type. Although two read queues are shown to receive read requests targeted at two different memory types, in other embodiments, another number of read queues may be used to receive read requests targeted at another number of different memory types. In addition, the queue 720 includes a first write queue 736 for storing received write requests targeted at the first memory type, and a second write queue 738 for storing received write requests targeted at the second memory type. In some embodiments, when one or more of the first memory type and the second memory type include a data storage technology that provides a miss status as a response to an access, the queue 720 also includes a miss queue 742. In one embodiment, one of the first memory type and the second memory type is an NVDIMM-P that provides a miss status response. In one embodiment, queue 720 includes queue 739 for storing scheduled memory access requests selected from one or more of queues 732-738 or a unified queue if a unified queue is used.

[0075] In some embodiments, the read scheduler 752 includes arbitration logic for selecting read requests out of order from the first read queue 732 and for selecting read requests out of order from the second read queue 734. In one embodiment, the read scheduler 752 selects a request from the first read queue 732 or the second read queue 734 when the corresponding request is available for scheduling in a given clock cycle from the first read queue 732 or the second read queue 734. In some embodiments, the read scheduler 752 schedules read requests to be transmitted out of order to one of the first memory type and the second memory type based on quality of service (QoS) or other priority information, age, process or thread identifier (ID), and relationship to other stored requests (such as targeting the same memory channel, targeting the same rank, targeting the same memory bank, and / or targeting the same page).

[0076] In order to avoid data conflicts on the memory data bus regardless of the multiple deterministic access wait times of the first memory type and the second memory type, in one embodiment, the read scheduler 752 determines the next given time point at which the memory data bus is available. In some embodiments, the time point is measured by clock cycles. The read scheduler 752 determines whether there is enough time to schedule the first memory access command corresponding to the selected read request stored in the first read queue 732 to provide response data at a given time point. In addition, the read scheduler 752 also determines whether there is enough time to schedule the second memory access command corresponding to the selected read request stored in the second read queue 734 to provide response data at a given time point. In other words, the read scheduler 752 determines whether a new memory access command received by the first read queue 732 or the second read queue 734 can be scheduled for transmission to the first memory device or the second memory device, so that a response to the new memory access command will be received on the memory data bus at a given time point. In various embodiments, the given time point is the next available time point at which the memory data bus is not scheduled to drive data on the memory data bus and has not yet been considered for scheduling.

[0077] In some embodiments, although the access latency of one or more of the first memory type and the second memory type is non-deterministic, the response has a deterministic latency. After the deterministic latency, the response is returned together with an indication indicating whether valid data is included in the response. If valid data is not included in the response, a retry is attempted later. Therefore, the memory access command is stored in the miss queue 742 for a later retry. As described earlier, other commands are sometimes used in addition to the memory access command. These other commands also add latency to the latency of the memory access command.

[0078] If there is enough time to transmit at least one of the first access command and the second access command to provide response data on the memory data bus at a given point in time, the read scheduler 752 selects one of the first memory access command and the second memory access command. The scheduler 752 may use the criteria described earlier, such as priority level, age, etc. In addition, a weighted value may be used. In one embodiment, the read scheduler 752 places the selected access command in the queue 739 before sending the selected access command to the corresponding memory type via the heterogeneous memory channel. In order to determine whether a new pending memory access command stored in either the first read queue 732 or the second read queue 734 can be scheduled for transmission at a given point in time, in one embodiment, the read scheduler 752 determines that the response waiting time of the new memory access command is N clock cycles, where N is an integer. The read scheduler 752 identifies an earlier time point corresponding to N clock cycles before the given time point, and determines whether the memory command bus is available at the earlier time point.

[0079] If the read scheduler 752 determines that there is enough time to schedule the above new memory access command, the read scheduler 752 schedules the new memory access command for transmission at an earlier time point and stores an indication that the memory data bus is not available at a given time point. In some embodiments, a bit vector is stored in a register to indicate which time points the memory data bus is available and which time points the memory data bus is not available. In one embodiment, a register is used to store indications indicating which time points have not yet been considered for scheduling and which time points have been considered for scheduling. In various embodiments, these stored indications can be used to determine other given time points for future scheduling commands for transmission.

[0080] In some embodiments, to avoid data conflicts on the memory data bus regardless of the multiple deterministic access wait times of the first memory type and the second memory type, the read scheduler 752 determines the next time point at which the memory command bus is available, compared to the next given time point at which the memory data bus is initially determined to be available. Likewise, in some embodiments, the time point is measured in clock cycles. In some embodiments, the read scheduler 752 determines the corresponding given time point for each different type of pending memory access command stored in the first read queue 732 and the second read queue 734 by adding the corresponding wait time to the next time point at which the memory command bus is available.

[0081] In order to determine whether a new pending memory access command stored in either the first read queue 732 or the second read queue 734 can be scheduled for transmission at the next time point when the memory command bus is available, in one embodiment, the read scheduler 752 determines the response latency of the new memory access command to be N clock cycles, where N is an integer. The read scheduler 752 identifies a later given time point corresponding to N clock cycles after the time point when the memory command bus is available. Thereafter, the read scheduler 752 determines whether the memory data bus is available at the later given time point.

[0082] In some embodiments, the read scheduler 752 uses the bit vector stored as described earlier to determine whether the memory data bus is available for each of the corresponding one or more given time points for each of the one or more pending memory access commands stored in the first read queue 732 and the second read queue 734. If the memory data bus is available for only a single pending memory access command during the corresponding given time point, the read scheduler 752 schedules the single pending memory access command at the next time point at which the memory command bus is available. If the memory data bus is available for multiple pending memory access commands during the corresponding given time point, the read scheduler 752 selects one of the pending memory access commands to transmit based on the criteria described earlier (such as priority level, age, etc.). The read scheduler 752 schedules the selected pending memory access command at the next time point at which the memory command bus is available.

[0083] The write scheduler 754 includes selection logic for the first write queue 736 and the second write queue 738 similar to that used by the read scheduler 752. In various embodiments, the write scheduler 754 also considers data conflicts caused by data driven on the shared memory data bus. The write scheduler 754 also uses the control logic used by the read scheduler 752 to implement the decision algorithm. In one embodiment, the response scheduler 756 includes similar logic for transmitting responses to the client out of order based on priority. In some embodiments, the received response data includes a tag or other identifier that the response scheduler 756 uses to identify which command is associated with the response data stored in the first read queue 732 or the second read queue 734. In other embodiments, the response scheduler 756 uses the arrival timing of the request data on the memory data bus to identify which command is associated with the response data. Therefore, although the request data arrives out of the order corresponding to the transmission of the command, the response scheduler 756 is able to track which received data belongs to which command.

[0084] In some embodiments, when the read scheduler 752 schedules a given command to be transmitted, the response scheduler 756 determines the given time point at which the requested read data is scheduled to arrive on the shared memory data bus. In one embodiment, the response scheduler 756 adds the waiting time of the given command to the time point at which the read scheduler 752 schedules to transmit the given command. In some embodiments, the response scheduler 756 generates an identifier. In some embodiments, the identifier is an indication of an entry in the request queue storing information corresponding to the given command. In other embodiments, the identifier is a combination of one or more of a part of the target address of the memory request corresponding to the given command with a thread identifier. The response scheduler 756 stores the association of the identifier with the given time point. In one embodiment, a table is used. Therefore, the response scheduler 756 can identify the given command based on the requested read data on the shared memory data bus at a given time point instead of based on the tag inserted in the given command or by the grouping associated with the requested read data arriving.

[0085] In some embodiments, control register 770 stores an indication of the current mode. For example, the off-chip memory data bus and the memory device support read mode or write mode at a given time. Therefore, traffic is routed in a given single direction during the current mode, and changes direction when the current mode is changed after the data bus turnaround latency. In various embodiments, control register 770 stores a threshold number (read burst length) of read requests to be sent during the read mode. In some embodiments, control register 770 stores the weight of the standard used by the selection algorithm for selecting the request stored in the queue 732-738 to transmit in the read scheduler 752 and the write scheduler 754.

[0086] refer to Figure 8 , shows one embodiment of a method 800 for scheduling memory requests for transmission to two different memory types. For discussion purposes, the embodiments of this embodiment (and Figures 9 to 12 However, it should be noted that in various embodiments of the described method, one or more of the described elements are performed simultaneously, in a different order than shown, or omitted entirely. Other additional elements may also be performed as needed. Any of the various systems or devices described herein are configured to implement method 800.

[0087] One or more clients within a node execute a computer program or software application. A computing resource determines that a given memory access request misses within a cache memory subsystem within a given client of the one or more clients. The client sends a memory access request to a system memory implemented by two different memories via a memory controller, the memory controller having a heterogeneous memory channel connected to each of the two different memories. A difference between one or more access latencies of a first type of memory and one or more access latencies of a second type of memory exceeds a threshold amount of time. Upon receiving a memory request for a first type of memory connected to a given memory channel, the memory request is stored (block 802). Upon receiving a memory request for a second type of memory connected to a given memory channel, the memory request is stored (block 804).

[0088] Based at least on the priority and the target of the memory request, memory requests to a first type of memory are marked to be transmitted out of order (block 806). Based at least on the priority and the target of the memory request, memory requests to a second type of memory are marked to be transmitted out of order (block 808). Thus, the memory controller supports out of order transmission for each of the first memory and the second memory. The memory requests are scheduled for transmission in a manner that provides response data at a given point in time (block 810). For example, the memory requests are scheduled in a hybrid manner without data conflicts on a shared memory data bus despite different access latencies.

[0089] In various embodiments, a scheduler or other control logic in a memory controller determines whether there are two pending memory access commands, such as a first command for a first memory type and a second command for a second memory type. The scheduler determines whether each of the first command and the second command can be transmitted without causing data conflicts on the shared memory data bus. For example, in addition to the access latency of each of the first type of memory and the second type of memory, based on the time point for transmitting the selected command in the first command and the second command, the memory controller tracks the time point at which the read response data or the write data is scheduled to arrive on the shared memory data bus. In some embodiments, the time point is measured in clock cycles.

[0090] If selecting either of the first command and the second command does not schedule a data conflict on the shared memory data bus, each of the first command and the second command remains a candidate for transmission. In this case, the scheduler selects a command from the first command and the second command based on arbitration logic. In other embodiments, determining whether to transmit the first command or the second command begins with selecting a specific given point in time to drive read response data or write data on the shared memory data bus.

[0091] Now turn to Fig. 9 , shows one embodiment of a method 900 for scheduling memory requests for transmission to two different memory types. A next given point in time at which read response data is driven on a memory data bus is identified (block 902). For example, both the access latency of each transmitted memory access command and status access command and the scheduled amount of requested data to be returned are considered in determining the next given point in time. In some embodiments, the point in time is measured in clock cycles.

[0092] If the read response data has been scheduled to arrive for a given point in time (the "yes" branch of conditional box 904), the control flow of method 900 returns to box 902, where the next given point in time is identified. For example, consider the next clock cycle after the currently selected clock cycle. Alternatively, a certain count is added to the current clock cycle, the count being equal to the given number of clock cycles consumed by scheduling the read request data to arrive from one of the two different memories. If the read response data is not scheduled to arrive for a given point in time (the "no" branch of conditional box 904), it is determined whether there is enough time to schedule a memory access command for a first memory type to provide response data at a given point in time (box 906). Next, it is determined whether there is enough time to schedule a memory access command for a second memory type different from the first memory type to provide response data at a given point in time (box 908).

[0093] In some embodiments, it is also determined whether there is enough time to schedule a status access command for the second memory type to provide response data at a given point in time (box 910). In some embodiments, the access latency of the status access command is different from the access latency of the memory access command for the second memory type. A command is selected from candidate commands that can provide response data at a given point in time (box 912). In various embodiments, the scheduler selects the next memory access command or status access command to transmit based on the criteria described earlier (such as priority level, age, etc.). The selected command is scheduled for transmission at a time point that allows the selected command to provide response data at a given point in time (box 914). For example, when scheduling memory access commands and status access commands for transmission, a scheduler of a memory controller with a heterogeneous memory channel considers possible additional commands and their corresponding wait times for preparing the selected command for transmission.

[0094] As described above, method 900 describes steps for avoiding data conflicts on a memory data bus regardless of multiple deterministic access latencies for a first memory type and a second memory type. However, as described earlier, in other embodiments, a scheduler of a memory controller having a heterogeneous memory channel determines a next time point at which a memory command bus is available, as compared to initially determining the next given time point at which the memory data bus is available. In some embodiments, the time point is measured in clock cycles. In some embodiments, the scheduler determines a corresponding given time point for each different type of pending memory access command by adding the corresponding latency to the next time point at which the memory command bus is available.

[0095] To determine whether a new pending memory access command can be scheduled for transmission at the next time point when the memory command bus is available, in one embodiment, the scheduler determines the response latency of the new memory access command to be N clock cycles, where N is an integer. The scheduler identifies a later given time point corresponding to N clock cycles after the time point when the memory command bus is available. Thereafter, the scheduler determines whether the memory data bus is available at the later given time point.

[0096] If the memory data bus is available for only a single pending memory access command during a respective given point in time, the scheduler schedules the single pending memory access command at the next point in time when the memory command bus is available. If the memory data bus is available for multiple pending memory access commands during a respective given point in time, the scheduler selects one of the pending memory access commands to transmit based on the criteria described earlier (such as priority level, age, etc.). The scheduler schedules the selected pending memory access command at the next point in time when the memory command bus is available.

[0097] The following description of methods 1000-1200 describes steps for initially determining the next given point in time at which the memory data bus is available, and then determining an earlier point in time at which the memory access command is scheduled to be transmitted on the memory command bus. However, in various other embodiments, as described above, the scheduler determines the next point in time at which the memory command bus is available, and then determines a later point in time at which the read response data is scheduled to arrive on the memory data bus without conflict. Although the steps in methods 900-1200 are described with respect to read access commands, in other embodiments, similar logic and steps are used for write access commands, where the write data is set on the shared memory data bus and may have data conflicts with read response data or other write data of other write access commands being driven on the shared memory data bus.

[0098] Now turn to Fig.10, shows one embodiment of a method 1000 for scheduling memory requests for transmission to two different memory types. In order to select an access command and schedule the access command for transmission at a time point that allows the selected access command to provide response data at a target given time point, a specific timing value is evaluated. In some embodiments, in ( Fig. 9 The following steps are performed after block 914 of method 900 of (hereinafter referred to as "the method 900"). A first amount of time between a given point in time for command transmission and a nearest point in time of a scheduled first access command for a first memory type is determined (block 1002). Wait times for any necessary additional commands to prepare for a possible next transmission of an access command for the first memory type are added to the first wait time for the access command for the first memory type (block 1004). Similar steps are performed for access commands for a second memory type. For example, a second amount of time between a given point in time and a nearest point in time of a scheduled second access command for a second memory type is determined (block 1006). Wait times for any necessary additional commands to prepare for a possible next transmission of an access command for the second memory type are added to the second wait time for the access command for the second memory type (block 1008).

[0099] A third amount of time between the given point in time and the nearest point in time of a scheduled third access command for the second memory type is determined (block 1010). Wait times for any necessary additional commands to prepare for a possible next transmission of the third access command for the second memory type are added to the third wait time for the third access command for the second memory type (block 1012). Each of the first amount of time, the second amount of time, and the third amount of time is compared to a corresponding one of the first wait time, the second wait time, and the third wait time (block 1014).

[0100] Now turn to Fig.11 , shows one embodiment of a method 1100 for scheduling memory requests for transmission to two different memory types. In order to select an access command and schedule the access command for transmission at a time point that allows the selected access command to provide response data at a target given time point, a specific comparison is made to the timing values. In some embodiments, in ( Fig.10 The following steps are performed after box 1014 of method 1000.

[0101] If the first wait time is not greater than the first amount of time (the "no" branch of conditional block 1102), then the first memory access command for the first memory type is inserted into the set of candidate commands for issuance (block 1104). In other words, if the cumulative wait time of the memory access command for the first memory type and any additional commands used to prepare the memory access command for issuance is less than or equal to the amount of time between the last issuance of any command for the first memory type and a given point in time, then there is sufficient time to issue the memory access command for the first memory type. For example, referring again to ( Figure 5 ) timing diagram 500, at least four clock cycles are required between transmitting "READ C" at CC 8 and completing transmitting "READ B" before CC 4.

[0102] If the first wait time is greater than the first amount of time (the "yes" branch of conditional block 1102), the first memory access command for the first memory type is removed from consideration as a candidate command for emission (block 1106). Similar steps are performed for a second memory access command for a second memory type. For example, if the second wait time is not greater than the second amount of time (the "no" branch of conditional block 1108), the second memory access command for the second memory type is inserted into the set of candidate commands for emission (block 1110). Otherwise, if the second wait time is greater than the second amount of time (the "no" branch of conditional block 1108), the second memory access command for the second memory type is removed from the set of candidate commands for emission (block 1112).

[0103] Similar steps are performed for a third memory access command for a second memory type. However, in some embodiments, a check is made to see whether the requested read data has been returned for the corresponding original memory access command. Again, briefly referring to ( Figure 4 4 ) timing diagram 400, since the requested read data is not returned for the original transaction read command "READ B", a read command "SREAD B" is transmitted. At CC 7, the requested read data is scheduled to arrive, but it is not returned from the second memory type. However, since the requested read data is returned at the scheduled given time point at CC 9, no subsequent read command is transmitted for the read command "READ D". In some embodiments, the memory controller receives an indication on another channel or link interface indicating whether the response data of the read access command is now available for a specific memory type in the first memory type and the second memory type. In other embodiments, the memory controller transmits a speculative read command to determine whether the response data is ready.

[0104] If the third wait time is not greater than the third amount of time (the "No" branch of conditional block 1114), and it is determined that the corresponding response data has not been returned (the "No" branch of conditional block 1116), the third memory access command for the second memory type is inserted into the candidate command set for transmission (block 1118). If the third wait time is not greater than the third amount of time (the "No" branch of conditional block 1114), and it is determined that the corresponding response data has been returned (the "Yes" branch of conditional block 1116), the third memory access command for the second memory type is removed from the candidate command set for transmission (block 1120). Similarly, if the third wait time is greater than the third amount of time (the "Yes" branch of conditional block 1114), the third memory access command for the second memory type is removed from the candidate command set for transmission (block 1120).

[0105] Now turn to Fig.12 , shows one embodiment of a method 1200 for scheduling memory requests for transmission to two different memory types. In order to select an access command and schedule the access command for transmission at a time point that allows the selected access command to provide response data at a target given time point, arbitration is performed between eligible candidate command sets. In some embodiments, in ( Fig.11 The following steps are performed after the steps of method 1100. Weights are assigned to criteria for selecting commands from a set of candidate commands for transmission (block 1202).

[0106] As described earlier, the criteria include one or more of: QoS or other priority information, age, process or thread identifier (ID), the amount of time since a memory access command was transmitted to the first memory type, and the amount of time since a memory access command or a status access command was transmitted to the second memory type. In some embodiments, the programmable control and status registers store weights assigned to the selected criteria. A candidate command set for transmission is determined (block 1204). In one embodiment, the commands are qualified after the steps of the previous methods 900-1100. If the set contains a single command (the "yes" branch of conditional block 1206), the single command is selected for transmission (block 1208).

[0107] If the set includes multiple commands (the "no" branch of conditional block 1206), a single command is selected from among the multiple candidates based on weighted criteria (block 1210). As described earlier, in some embodiments, the scheduler assigns a given weight to each of the criteria and performs a weighted sum. The memory access command or state access command with the largest sum is selected for transmission. The total latency of the selected command, including any necessary additional commands to prepare for the transmission of the selected command, is subtracted from the time point at which the read response data is scheduled to arrive (block 1212). The additional commands and the selected access command are scheduled to be transmitted at the determined time point found by performing the subtraction (block 1214).

[0108] In various embodiments, the previously described methods and / or mechanisms are implemented using program instructions of a software application. The program instructions describe the behavior of the hardware in a high-level programming language (such as C). Alternatively, a hardware design language (HDL) such as Verilog is used. The program instructions are stored on a non-transitory computer-readable storage medium. Various types of storage media are available. The storage medium can be accessed by a computing system during use to provide the computing system with program instructions and accompanying data for program execution. The computing system includes at least one or more memories and one or more processors configured to execute the program instructions.

[0109] It should be emphasized that the above embodiments are only non-limiting examples of implementations. Once the above disclosure is fully understood, many changes and modifications will become apparent to those skilled in the art. The following claims are intended to be interpreted as including all such changes and modifications.

Claims

1. A memory controller comprising: a first interface, the first interface being used to receive a memory request; A second interface, wherein the second interface comprises: a command bus for sending a memory access command corresponding to the memory request to a first memory device or a second memory device different from the first memory device; and a data bus for transmitting data between the memory controller and both the first memory device and the second memory device; and control logic, wherein in response to determining that the data bus is scheduled to be available at a first point in time, the control logic is configured to determine whether a new memory access command can be scheduled for transmission to one of the first memory device and the second memory device such that a response to the new memory access command will be received on the data bus at the first point in time; Wherein the first time point is earlier than a second time point at which a response to an outstanding memory access command transmitted to the other of the first memory device and the second memory device will be received on the data bus. 2 . The memory controller of claim 1 , wherein the first point in time is a next available point in time where the data bus is not scheduled to have data driven on the data bus and has not yet been considered for scheduling.

3. The memory controller of claim 1 , wherein to determine whether the new memory access command can be scheduled for issuance, the control logic is configured to: Determine that the response waiting time of the new memory access command is N clock cycles, where N is an integer; and identifying a third time point corresponding to N clock cycles before the first time point; and It is determined whether the command bus is available at the third time point.

4. The memory controller of claim 3, wherein in response to determining that there are two or more pending memory access commands with the response latency, the control logic is configured to schedule one of the two or more pending memory access commands to be transmitted at the third time point.

5. The memory controller of claim 3 , wherein in response to determining that there is sufficient time to schedule the new memory access command, the control logic is configured to: scheduling the new memory access command to be transmitted at the third time point; and An indication that the data bus is unavailable at the first point in time is stored.

6. The memory controller of claim 3 , wherein in response to determining that there are two or more pending memory access commands with the response latency, the control logic is configured to schedule one of the two or more pending memory access commands at the third point in time based on one or more of: a first amount of time between the first point in time and a most recent point in time at which a memory access command was issued to the first memory device; and A second amount of time between the first point in time and a most recent point in time at which a memory access command was issued to the second memory device.

7. The memory controller of claim 6, wherein the control logic is further configured to select one of the two or more pending memory access commands to transmit based on one or more of: The priority levels of the first memory access command and the second memory access command; and The age of the first memory access command and the second memory access command.

8. The memory controller of claim 6, wherein in response to determining that a given memory access command having the response latency is pending, the control logic is further configured to: determining that the given memory access command is a state access command targeting the same address as an earlier issued memory access command accessing the second memory device; and It is determined that read response data of the earlier issued memory access command has not arrived.

9. A method comprising: Receiving a memory request by a first interface; Sending, by the second interface via a command bus, a memory access command corresponding to the memory request to a first memory device or a second memory device different from the first memory device; transmitting data between a memory controller and both the first memory device and the second memory device via a data bus by the second interface; responsive to determining that the data bus is scheduled to be available at a first point in time, determining whether a new memory access command can be scheduled for transmission to one of the first memory device and the second memory device such that a response to the new memory access command will be received on the data bus at the first point in time; Wherein the first time point is earlier than a second time point at which a response to an outstanding memory access command transmitted to the other of the first memory device and the second memory device will be received on the data bus.

10. The method of claim 9, wherein the first point in time is a next available point in time where the data bus is not scheduled to have data driven on the data bus and has not yet been considered for scheduling.

11. The method of claim 9, wherein in order to determine whether the new memory access command can be scheduled for transmission, the method further comprises: Determine that the response waiting time of the new memory access command is N clock cycles, where N is an integer; as well as Identify a third time point corresponding to N clock cycles before the first time point; as well as It is determined whether the command bus is available at the third time point.

12. The method of claim 11 , wherein in response to determining that there are two or more pending memory access commands having the response latency, the method further comprises: One of the two or more pending memory access commands is scheduled to be transmitted at the third point in time.

13. The method of claim 11 , wherein in response to determining that there is sufficient time to schedule the new memory access command, the method further comprises: scheduling the new memory access command to be transmitted at the third time point; as well as An indication that the data bus is unavailable at the first point in time is stored.

14. The method of claim 11, wherein in response to determining that there are two or more pending memory access commands having the response latency, the method further comprises: Scheduling one of the two or more pending memory access commands at the third point in time based on one or more of: a first amount of time between the first point in time and a most recent point in time at which a memory access command was issued to the first memory device; as well as A second amount of time between the first point in time and a most recent point in time at which a memory access command was issued to the second memory device.

15. The method of claim 14, wherein the method further comprises: One of the two or more pending memory access commands is selected to be transmitted based on one or more of: The priority levels of the first memory access command and the second memory access command; and The age of the first memory access command and the second memory access command.

16. The method of claim 14, wherein in response to determining that a given memory access command having the response latency is pending, the method further comprises: determining that the given memory access command is a state access command that targets the same address as an earlier issued memory access command that accessed the second memory device; as well as It is determined that read response data of the earlier issued memory access command has not arrived.

17. A computing system comprising: a first memory device; a second memory device, the second memory device being different from the first memory device; a processor configured to generate a memory access request for data stored in the first memory device or the second memory device; a communication fabric configured to transmit data; as well as a memory controller coupled to the processor via the communication fabric, and coupled to each of the first memory device and the second memory device, wherein the memory controller is configured to: sending a memory access command corresponding to the memory access request to the first memory device or the second memory device via a command bus; transferring data between the memory controller and both the first memory device and the second memory device via a data bus; responsive to determining that the data bus is scheduled to be available at a first point in time, determining whether a new memory access command can be scheduled for transmission to one of the first memory device and the second memory device such that a response to the new memory access command will be received on the data bus at the first point in time; Wherein the first time point is earlier than a second time point at which a response to an outstanding memory access command transmitted to the other of the first memory device and the second memory device will be received on the data bus.

18. The computing system of claim 17, wherein the first point in time is a next available point in time where the data bus is not scheduled to have data driven on the data bus and has not yet been considered for scheduling.

19. The computing system of claim 17, wherein to determine whether the new memory access command can be scheduled to be transmitted at the first point in time, the memory controller is configured to: Determine that the response waiting time of the new memory access command is N clock cycles, where N is an integer; and identifying a third time point corresponding to N clock cycles before the first time point; and It is determined whether the command bus is available at the third time point.

20. The computing system of claim 19, wherein in response to determining that there is sufficient time to schedule the new memory access command, the memory controller is configured to: scheduling the new memory access command to be transmitted at the third time point; and An indication that the data bus is unavailable at the first point in time is stored.

Images