Method and apparatus for restoring normal access performance in fine-grained DRAMs - Patents.com

Abstract

A fine-grained dynamic random access memory (DRAM) includes a first memory bank, a second memory bank, and a dual mode I / O circuit. The first memory bank includes a memory array divided into a plurality of grains, each grain including a row buffer and an input / output (I / O) circuit. The dual mode I / O circuit is coupled to the I / O circuit of each grain in the first memory bank and operates in a first mode in which commands having a first data width are routed to each grain and executed individually at each grain, and in a second mode in which commands having a second data width different from the first data width are executed in parallel by at least two of the grains.

Description

[Background technology]

[0001] High-bandwidth dynamic random-access memory (DRAM) is used by Graphics Processing Units (GPUs) and other throughput-oriented parallel processors. Next-generation GPU-specific DRAMs are primarily optimized to maximize bandwidth, first by using high-frequency off-chip signaling such as that used with graphics double data rate memory, and more recently through on-package integration of wide, high-bandwidth interfaces to the processor die and stacks of DRAM chips such as those used in High Bandwidth Memory (HBM / HBM2) modules.

[0002] Many important workload classes for such systems experience irregular, fine-grained memory accesses. Such workloads include graph traversal (widely used in social networking, search, e-commerce, cloud, and gig economy), key-value store access, and high performance computing (HPC). Next-generation grain-based stacked DRAMs are critical to enable energy-efficient, high-bandwidth fine-grained irregular accesses.

[0003] Fine-grained DRAM (FG-DRAM) allows stacked DRAM memories such as HBM to scale memory bandwidth without exceeding practical Thermal Design Power (TDP) limits. FG-DRAM achieves such scaling by partitioning the DRAM die into smaller independent units (called grains), each with dedicated DQ pins rather than command-address (CA) pins. For example, a traditional DRAM bank can be split into two to achieve two grains, each with half the number of columns compared to the original bank. The result is a significant increase in memory-level parallelism. Additionally, a significant reduction in activation energy is realized.

[0004] However, in such grain-based architectures, the access latency of DRAM atoms via column access commands to open rows increases linearly with the number of grains per bank, as each grain has a narrower interface to the DRAM channel. Such an increase is detrimental to latency-sensitive normal access workloads. [Brief description of the drawings]

[0005] [Figure 1] 1 is a block diagram of an accelerated processing unit (APU) and memory system known in the prior art. [Diagram 2] 2 is a block diagram of a memory controller suitable for use in an APU similar to FIG. 1 , according to some embodiments. [Diagram 3] FIG. 1 is a block diagram comparing two memory banks of two DRAMs showing grain usage. [Figure 4] FIG. 2 is a block diagram showing an address line arrangement for a fine-grained DRAM memory interleaved in a grain-independent format. [Diagram 5] FIG. 2 is a block diagram showing an address line arrangement for a fine-grained DRAM memory interleaved in a grain stripe format. [Figure 6] FIG. 1 is a block diagram illustrating a fine-grained memory bank according to the prior art. [Figure 7] FIG. 2 is a block diagram illustrating a fine-grained memory bank, according to some embodiments. [Figure 8] FIG. 13 is a block diagram illustrating a fine-grained memory bank, in accordance with some additional embodiments. [Figure 9] FIG. 2 is a flow diagram of a process for operating a fine-grained memory according to some embodiments. [Figure 10] 1 is a block diagram illustrating a data processing system including library instructions for operating fine-grained memory in accordance with some embodiments. [Figure 11] FIG. 2 is a flow diagram of a process for operating a fine-grained memory according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] In the following description, the use of the same reference numbers in different figures indicates similar or identical items. Unless otherwise noted, the word "coupled" and its related verb forms include both direct and indirect electrical connections by means well known in the art, and unless otherwise noted, any description of a direct connection also refers to alternative embodiments using a suitable form of indirect electrical connection.

[0007] A fine-grained dynamic random access memory (DRAM) includes a first memory bank, a second memory bank, and a dual mode I / O circuit. The first memory bank includes a memory array divided into a plurality of grains, each grain including a row buffer and an input / output (I / O) circuit. The dual mode I / O circuit is coupled to the I / O circuit of each grain in the first memory bank and operates in a first mode in which commands having a first data width are routed to each grain and executed individually at each grain, and in a second mode in which commands having a second data width different from the first data width are executed in parallel by at least two of the grains.

[0008] A method of operating a volatile memory includes sending a column address strobe (CAS) command from a memory controller to a first grain in a first bank on the volatile memory. The method includes sending a multicast CAS command to dual mode I / O circuitry on the volatile memory. In response to the multicast CAS command at the volatile memory, the method includes sending a CAS signal to a plurality of grains including the first grain.

[0009] The data processing system includes a plurality of processing units, a data fabric coupled to the plurality of processing units, a memory controller coupled to the data fabric for servicing memory access requests from the processing units, and a fine-grained dynamic random access memory (DRAM) in communication with the memory controller. The fine-grained DRAM includes a first memory bank, a second memory bank, and a dual mode I / O circuit. The first memory bank includes a memory array divided into a plurality of grains, each grain including a row buffer and an input / output (I / O) circuit. The dual mode I / O circuit is coupled to the I / O circuit of each grain in the first memory bank, and the dual mode I / O circuit operates in a first mode in which commands having a first data width are routed to each grain and executed individually at each grain, and in a second mode in which commands having a second data width different from the first data width are executed in parallel by at least two of the grains.

[0010] 1 is a block diagram of an accelerated processing unit (APU) 100, according to some embodiments. The APU 100 is implemented as a system-on-chip (SoC) that may be part of a variety of host data processing platforms. Although an APU is shown in this embodiment, other data processing platforms such as a central processing unit (CPU) or a graphics processing unit (GPU) may be used. For example, in some embodiments, the fine-grained memory access techniques herein are embodied in a GPU chip employed in a graphics card or other graphics processing module. In other embodiments, a dedicated processor core such as an intelligence processing unit (IPU) may be used. In this embodiment, APU 100 generally includes a CPU core complex 110, a graphics core 120, a set of display engines 130, a memory management hub 140, a data fabric 150, a set of peripheral controllers 160, a set of peripheral bus controllers 170, a system management unit (SMU) 180, flash memory 205, and a set of FG_DRAM memory controllers 190.

[0011] CPU core complex 110 includes CPU core 112 and CPU core 114. In this example, CPU core complex 110 includes two CPU cores, but in other embodiments, CPU core complex 110 may include any number of CPU cores. Each of CPU cores 112 and 114 is bidirectionally connected to a system management network (SMN) 145 and a data fabric 150 that form a control fabric, and can provide memory access requests to data fabric 150. Each of CPU cores 112 and 114 may be a single core or may be a core complex having two or more single cores that share certain resources, such as a cache.

[0012] Each of the graphics cores 120 is a high-performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, texture blending, etc., in a highly integrated and parallel manner. Each of the graphics cores 120 is bidirectionally connected to the SMN 145 and to the data fabric 150 and can provide memory access requests to the data fabric 150. In this regard, the APU 100 can support either a unified memory architecture in which the CPU core complex 110 and the graphics core 120 share the same memory space, or a memory architecture in which the CPU core complex 110 and the graphics core 120 share a portion of the memory space, but the graphics core 120 also uses a private graphics memory that is not accessible by the CPU core complex 110.

[0013] Display engine 130 renders and rasterizes objects generated by graphics core 120 for display on a monitor. Graphics core 120 and display engine 130 are bidirectionally connected to a common memory management hub 140 for uniform translation to appropriate addresses in memory, which in turn is bidirectionally connected to data fabric 150 for generating such memory accesses and receiving read data returned from the memory system.

[0014] Data fabric 150 includes a crossbar switch for routing memory access requests and responses between any memory accessing agent and memory controller 190. The data fabric also includes a system memory map defined by the basic input / output system (BIOS) and buffers for each virtual connection to determine where to send memory accesses based on the system configuration.

[0015] Peripheral controllers 160 include a USB controller 162 and a serial advanced technology attachment (SATA) interface controller 164, each of which is bidirectionally connected to a system hub 166 and to SMN 145. These two controllers are merely examples of peripheral controllers that may be used in APU 100.

[0016] Peripheral bus controller 170 includes system controller hub 172 and peripheral controller hub 174, each of which is bidirectionally connected to input / output (I / O) hub 176 and SMN 145. System controller hub 172 connects to flash memory 205 via appropriate communications links. I / O hub 176 is bidirectionally connected to system hub 166 and to data fabric 150. Thus, for example, a CPU core can program registers in USB controller 162, SATA interface controller 164, system controller hub 172, or peripheral controller hub 174, with accesses routed by data fabric 150 through I / O hub 176.

[0017] SMU 180 is a local controller that controls the operation of resources on APU 100 and synchronizes communication between them. SMU 180 manages the power-up sequencing of the various processors on APU 100 and controls multiple off-chip devices via reset, enable, and other signals. SMU 180 also manages the power of the various processors and other functional blocks.

[0018] Although an SoC embodiment is shown, this is not limiting and other computing platforms can benefit from the techniques described herein.

[0019] Figure 2 is a block diagram of a memory controller 200 suitable for use in an APU such as Figure 1. The memory controller 200 generally includes an interface 212, a memory interface queue 214, a command queue 220, an address generator 222, a content addressable memory (CAM) 224, a replay control logic 231 including a replay queue 230, a refresh control logic 232, a timing block 234, a page table 236, an arbiter 238, an error correction code (ECC) check circuit 242, an ECC generation block 244, and a data buffer 246.

[0020] Interface 212 has a first bidirectional connection to the data fabric via an external bus and has an output. In memory controller 200, this external bus conforms to the Advanced Extensible Interface version 4 specified by ARM Holdings, PLC of Cambridge, UK, known as "AXI4", but may be other types of interfaces in other embodiments. Interface 212 translates memory access requests from a first clock domain known as the FCLK (or MEMCLK) domain to a second clock domain internal to memory controller 200, known as the UCLK domain. Similarly, memory interface queue 214 provides memory access from the UCLK domain to the DFICLK domain associated with the DFI interface.

[0021] Address generator 222 decodes addresses of memory access requests received from the data fabric via the AXI4 bus. The memory access requests include access addresses in the physical address space represented in a normalized format. Address generator 222 converts the normalized addresses into a format that can be used to address actual memory devices in the memory system and to efficiently schedule the associated accesses. This format includes a region identifier that associates the memory access request with a particular rank, row address, column address, bank address, and bank group. At power-up, the system BIOS queries the memory devices in the memory system to determine their size and configuration, and programs a set of configuration registers associated with address generator 222. Address generator 222 converts the normalized addresses into the appropriate format using the configuration stored in the configuration registers. Command queue 220 is a queue of memory access requests received from memory access agents in APU 100, such as CPU cores 112, 114 and graphics core 120. Command queue 220 stores address fields decoded by address generator 222 and other address information that allows arbiter 238 to efficiently select memory accesses including access type and quality of service (QoS) identifiers. CAM 224 contains information for enforcing ordering rules, such as write after write (WAW) and read after write (RAW) ordering rules. Command queue 220 is a stacked command queue that contains multiple entry stacks, each containing multiple command entries, in this embodiment 32 entry stacks of four entries each, as described further below.

[0022] An error correction code (ECC) generation block 244 determines the ECC for the write data being sent to memory. This ECC data is then added to the write data in a data buffer 246. An ECC check circuit 242 checks the received ECC against the incoming ECC.

[0023] Replay queue 230 is a temporary queue for storing memory accesses selected by arbiter 238 waiting for responses, such as address and command parity responses. Replay control logic 231 accesses ECC check circuitry 242 to determine whether the returned ECC is correct or indicates an error. Replay control logic 231 initiates and controls a replay sequence in which accesses are replayed in the event of a parity or ECC error in any of these cycles. Replayed commands are placed in memory interface queue 214.

[0024] The refresh control logic 232 includes state machines for various power-down, refresh, and termination resistor (ZQ) calibration cycles that are generated separately from normal read and write memory access requests received from memory access agents. For example, when a memory rank is in precharge power-down, the refresh logic must be periodically woken up to perform refresh cycles. The refresh control logic 232 periodically generates refresh commands in response to defined conditions to prevent data errors caused by charge leakage from storage capacitors of memory cells in a DRAM chip. The refresh control logic 232 includes an activation counter 248, which in this embodiment has a counter for each memory region that counts the rolling number of activation commands sent to the memory region over the memory channel. The memory region is a memory bank in some embodiments and a memory sub-bank in other embodiments, as further described below. Additionally, the refresh control logic 232 periodically calibrates the ZQ to prevent on-die termination resistor mismatches due to thermal changes in the system.

[0025] Arbiter 238 is bidirectionally connected to command queue 220 and is the heart of memory controller 200, performing intelligent scheduling of accesses to improve memory bus utilization. In this embodiment, arbiter 238 includes grain address translation block 240 for translating commands and addresses for fine-grained memory access in multiple modes, as described further below. In some embodiments, such functionality may be performed by address generation block 222. Arbiter 238 uses timing block 234 to enforce the proper timing relationships by determining whether a particular access in command queue 220 is eligible for issue based on DRAM timing parameters. For example, each DRAM may have a "t RCA timing block 234 maintains a set of counters that determine eligibility based on this and other timing parameters defined in the JEDEC specifications, and is bidirectionally connected to the replay queue 230. A page table 236 maintains state information regarding active pages in each bank and rank of the memory channel for an arbiter 238, and is bidirectionally connected to the replay queue 230. The arbiter 238 contains a single command input for each entry stack of the command queue 220, from which it selects and schedules commands for dispatch to the DRAM channel via the memory interface queue 214.

[0026] In response to a write memory access request received from interface 212, ECC generation block 244 calculates an ECC according to the write data. Data buffer 246 stores the write data and ECC for the received memory access request. The data buffer outputs the combined write data / ECC to memory interface queue 214 when arbiter 238 selects the corresponding write access for dispatch to the memory channel.

[0027] The memory controller 200 includes circuitry that allows it to select memory accesses for dispatch to the associated memory channel. To make the desired arbitration decisions, the address generator 222 decodes the address information into pre-decoded information including rank, row address, column address, bank address, and bank group within the memory system, and the command queue 220 stores the pre-decoded information. Configuration registers (not shown) store configuration information for determining how the address generator 222 decodes the address information it receives. The arbiter 238 uses the decoded address information, timing eligibility information indicated by the timing block 234, and active page information indicated by the page table 236 to efficiently schedule memory accesses while adhering to other criteria such as quality of service (QoS) requirements. For example, the arbiter 238 implements prioritization of accesses to open pages and hides overhead accesses to one bank by interleaving them with read and write accesses to another bank to avoid the overhead of precharge and activate commands required to change memory pages. In particular, during normal operation, the arbiter 238 typically keeps pages open in different banks until those pages need to be precharged before selecting a different page.

[0028] 3 is a block diagram 300 comparing two memory banks 302 and 304 of two DRAMs illustrating the use of grains. DRAM bank 302 is constructed without grains, while DRAM bank 304 includes two grains 306 and 308. DRAM bank 304 is part of a larger DRAM integrated circuit that includes multiple DRAM banks. DRAM bank 304 generally includes a memory array divided into multiple grains, in this case two grains, with each grain including row buffers and input / output (I / O) circuitry.

[0029] The conventional non-grained architecture of DRAM bank 302 can perform 32-byte (32B) memory accesses over 32 data lines DQ[0:31] accessed in 8 bursts or "beats" on a pseudo-channel (pCH) assigned to DRAM bank 302. Issuing a column command to an open row in the bank provides 32 bytes (32B) of data in 8 bursts because the bank is accessing all 32 DQ pins of the pseudo-channel, and the pins are not split, providing a 2x faster data read compared to the 2-grained architecture of DRAM bank 304. As a result, the time to read a 64B cache line from an open row in a non-grained architecture is 2x faster than the 2-grained architecture of DRAM bank 304. * It takes 8=16 bursts plus one additional long column-to-column delay (tCCDL) minus the burst duration on the data bus (tBURST) for overhead time.

[0030] When using a fine-grained architecture without the techniques herein, the DRAM bank 304 is accessed in 16B accesses via two designated sets of pCH data lines, DQ[0:15] and DQ[16:31]. In this exemplary configuration with 16 DQ pins of the pseudo channel dedicated to the grain, an open row within the grain responds to a column command by providing 16B of data in eight bursts. Thus, the time to read a 64B cache line from an open row is 4. * It takes 8=32 bursts plus an additional 3 (tCCDL-tBURST) cycles of overhead. Thus, the grain architecture incurs a 33% latency penalty for each individual 32B access.

[0031] FIG. 4 is a diagram illustrating an address line arrangement for a fine-grained DRAM memory interleaved in a grain-independent format. In the illustrated arrangement, address line numbers are shown at the top with address lines labeled below each address line number. "RO" stands for row, "CO" stands for column, "GR" stands for grain number, "BK" stands for bank number, "BG" stands for bank group number, "PCH" stands for pseudo channel, and "X" indicates that the address line is unassigned. In grain-independent address interleaving, no striping is performed to allow for irregular throughput-oriented 16B accesses, and the grain address bits of address line 12 are used in the same way as the bank address bits.

[0032] FIG. 5 illustrates an address line arrangement for a fine-grained DRAM memory interleaved in grain stripe format. The illustrated embodiment deploys modified address interleaving in the memory controller to achieve a 32B arrangement of grain stripes with grain stripe address interleaving. 32B of data is split into two correlated grains of 16B each. The GR address bit is present on the lowest address line (line 4), meaning that adjacent grains are addressed by this bit. Note that this approach can be extended to support an increased number of grains per bank, e.g., 4 grains, 8 grains, or 16 grains, etc. (theoretically limited only by the number of columns in the bank).

[0033] However, simply using the grain stripe address interleaving of FIG. 5 alone suffers from an additional 75% overhead that is unacceptable due to the tCCDL constraint that exists between column commands to the same physical bank. This overhead can be lowered in some embodiments to an overhead of tCCDS by separating "partner" grains (grains that serve adjacent addresses) into different bank groups. However, even in such implementations, the overhead is still not negligible. Furthermore, such separation of partner grains into different bank groups imposes additional scheduling and data response reconfiguration complexities on the memory controller to avoid asynchronous latencies between the two grains.

[0034] FIG. 6 is a block diagram of a fine-grained memory bank 600 according to the prior art, and FIG. 7 is a block diagram of a fine-grained memory bank 700 according to some embodiments.

[0035] Memory bank 600 includes two grains, labeled "Grain 0" and "Grain 1," a row decoder 602, two column decoders 604 and 606, a grain select circuit 608 labeled "Gr.Sel," a command bus 620, and an address bus 622. In operation, memory bank 600 is accessed with commands, such as the illustrated column address strobe (CAS) command, sent from a memory controller over command bus 620. As shown, address bus 622 provides column address bits (CO) to column decoders 604 and 606 and grain bits (GR) to grain selector circuit 608 according to the addressing scheme of FIG. 4. Memory accesses are one grain wide, which in this example is 16B.

[0036] 7, memory bank 700 includes two grains, labeled "grain 0" and "grain 1," and I / O circuitry including a row decoder 702, two column decoders 704 and 706, a row buffer (not shown separately) in each grain, a dual mode input / output (I / O) circuit 708 labeled "mCAS supp," a command bus labeled 720, and an address bus 722. In this embodiment, dual mode I / O circuit 708 is connected to the I / O circuitry of each grain in the memory bank.

[0037] The dual mode I / O circuit 708 includes a first input labeled "mCAS enable," a second input labeled "grain bit," a first output labeled "grain 0 enable," and a second output labeled "grain 1 enable." The first input receives a multicast CAS enable signal from the memory controller over a command bus 720, and the second input receives the grain address bit (GR, FIG. 5). The first output is connected to a column decoder 704, and the second output is connected to a column decoder 706.

[0038] The dual mode I / O circuit 708 generally operates in a first mode in which a command having a first data width (16B) is routed to each grain and executed individually at each grain, and in a second mode in which a command having a second data width (32B) different from the first data width is executed in parallel by at least two of the grains. Such a configuration overcomes the overhead imposed by the tCCDL constraint described above by using a multicast column address strobe (mCAS) command for two or more partnered grains, both of which are grains of bank 700 in this example. The address bits that make up the mCAS command are identical for both grains, except for the grain identifier bits. Thus, the illustrated embodiment augments the shared command and address (CA) path / logic with support for the mCAS command.

[0039] In the illustrated two-grain example, the dual mode I / O circuit 708 is implemented using a multiplexer augmented with an mCAS enable signal, although other logic implementations are employed in other embodiments. The dual mode I / O circuit 708 is shown in more detail on the right side of the figure and in this embodiment includes a multiplexer 710 and two OR gates 712. The multiplexer 710 has an input that receives a grain address bit and two outputs connected to respective inputs of the OR gates 712. Each OR gate 712 has a first input connected to the output of the multiplexer 710 and a second input that receives a multicast CAS enable signal.

[0040] In operation, when the multicast CAS enable signal is HIGH to signal that the current CAS command is a multicast CAS command, the output of OR gate 712 goes HIGH, enabling the multicast CAS command by asserting CAS for selected columns in both grains of bank 700. The grains then respond to the CAS in parallel to provide a higher data width. In this mode, both grains receive an mCAS command and both respond in parallel with 16B of data read from their respective addresses presented on the address bus. When the multicast CAS enable signal is low, only one of the OR gates 712 has a high output at a time, allowing only one grain at a time to receive the CAS command, thus providing normal fine-grained operation.

[0041] While the bank 700 in this embodiment has two grains, higher grain counts are provided in some embodiments including a multi-level multiplexer tree in the dual mode I / O circuitry 708. For example, in one exemplary embodiment with eight grains in the bank 700, each grain can provide fine-grained memory access having a width of 4B in a first mode, while all eight grains operating in parallel provide access having a width of 32B in a second mode. The dual mode I / O circuitry 708 in such an embodiment includes a three-level multiplexer tree that receives three grain bits. In the first mode, the multiplexer tree routes the CAS command to the appropriate grain being addressed, activating only one of the eight column decoders. In the second mode, the dual mode I / O circuitry causes the CAS command to be asserted at each grain in response to a multicast CAS command by enabling all eight column decoders.

[0042] 8 is a block diagram of a fine-grained memory bank 800 according to some additional embodiments. The memory bank 800 includes two grains, labeled "Grain 0" and "Grain 1," a row decoder 802, two column decoders 804 and 806, a row buffer in each grain (not shown separately), a grain selection circuit 808, labeled "Gr.Sel," a multicast CAS relay circuit 810, labeled "mCAS Relay," and I / O circuitry including a command bus 820 and an address bus 822.

[0043] In this embodiment, the dual mode I / O circuit is embodied as a multicast CAS relay circuit 810 rather than a multiplexer tree implementation as in FIG. 7. When there are many grains per bank (e.g., number of grains "G", such as 8 or 16), the additional depth of the multiplexer tree implementation causes an increase in critical path command latency. Instead of using a multiplexer tree, the illustrated embodiment uses a multicast CAS relay circuit 810 to perform localized asynchronous repetition (relay) of the CAS signal. The multicast CAS relay circuit 810 is co-located with the column decoder 804 for grain 0 and includes a first input that receives the mCAS enable signal and an output connected to the column decoder for grain 1. In an embodiment with more than two grains, the multicast CAS relay circuit 810 is present in the I / O circuit for multiple grains to forward the mCAS to other grains. The structure of the multicast CAS relay circuit 810 has a lower capacitance than a multiplexer tree implementation due to its shorter electrical conductors, and therefore does not increase command latency as much when there are many grains.

[0044] In operation, the memory bank 800 is accessed with commands, such as the illustrated column address strobe (CAS) command, sent over a command bus from the memory controller. As shown, the address bus carries column address bits (CO in FIG. 4) to column decoders 804 and 806 and grain bits (GR in FIG. 4) to grain selector circuit 808 according to the addressing scheme of FIG. 4. In a first mode, memory accesses are one grain wide, in this example 16B. In a second mode, memory accesses are two or more grains wide (two in this example, totaling 32B). The second mode is activated by an mCAS enable signal provided from the memory controller to the multicast CAS relay circuit 810. In the second mode, the multicast CAS relay circuit 810 performs a localized relay of the multicast CAS command in the second mode to forward it to another one of the at least two grains by replacing the initial grain identifier value in the multicast CAS command with a new value that identifies the other grain. In some embodiments, the multicast CAS relay circuit 810 identifies the new value using a lookup table or an adder.

[0045] Although two different embodiments for a dual mode I / O circuit are described with respect to FIGS. 7 and 8, other suitable digital logic schemes may be implemented in the memory to perform the steps of operating in the first and second modes.

[0046] 9 is a flow diagram 900 of a process for operating a fine-grained memory, according to some embodiments. In this embodiment, the process uses three variations of the memory load (read) command listed on the right, including a "LD" command that uses a 32B request to main memory in case of a cache miss, a "LD_x" command that specifies an irregular load size "x" serviceable from an on-chip scratchpad rather than from main memory, and a "LD_x_MEM" command that specifies an irregular load of size x serviceable from main memory via a single 16B request.

[0047] As shown in block 902, the process includes tagging the selected LD memory access request as a LD_x_MEM request if the selected LD memory access request is an irregular request that must cause a 16B fetch from main memory. The tagging in block 902 may be performed by a programmer generating the original software from which the memory access command occurs, or by a compiler or just-in-time compiler that interprets programming instructions for a specified memory system.

[0048] At block 904, the memory controller receives a memory access request associated with the tagged memory access request, for example, when a program or library executes an instruction tagged at block 902. The request at this point can have a first data width, such as 16B as shown, or a second data width, such as 32B as shown. At block 906, the process determines whether the request has a first data width, for example, 16B. If so, the process proceeds to block 908, where the process applies grain-independent address interleaving to the request to serve the request with 16B-wide fine-grained memory access. The memory controller then issues a CAS command to the 16B-wide memory grain to execute the command. As described above, the request is served by an access to a single grain of memory in the respective memory bank in the first mode. When an activate is issued for the 16B CAS at block 910, the memory controller receives response data from the memory in eight bursts of 16b.

[0049] Referring again to block 906, if the request is not a 16B request, the process proceeds to blocks 912-914 to service the request using the dual mode I / O circuitry operating in the second mode as described above. In block 912, the 32B request is split into two 16B sub-requests that are associated with each other according to a grain stripe arrangement, for example as shown in FIG. 5. In block 914, address interleaving for such grain stripe scheme is applied to the two 16B commands, and a 32B multicast CAS command is issued from the memory controller to execute both commands from the memory. In response to the multicast CAS command, the memory services the request by asserting column address strobes in parallel for selected columns in at least two grains. For example, the dual mode I / O circuitry of 708 (FIG. 7), or the multicast relay circuitry 808 (FIG. 8), operating in the second mode, asserts the multicast CAS command in multiple memory grains. The resulting data is returned to the memory controller in two parallel bursts of 16B width, as shown in block 916. The memory controller may also perform collation of the received data to provide the proper order of data in response to the original memory request.

[0050] 10 is a block diagram of a data processing system 1000 including library instructions for operating fine-grained memory, according to some embodiments. The data processing system 1000 includes an operating system (OS) 1002, a memory controller 1008, and a memory 1010. The operating system 1002 executes on one or more processors of the data processing system and includes an instruction set architecture (ISA) command library 1004 and a just-in-time (JIT) compiler 1006.

[0051] In some embodiments, the extended ISA command library 1004 includes the load commands LD_x and LD_x_MEM, as described above with respect to FIG. 9, in addition to the standard LD commands and other commands in the commonly used ISA library as indicated by the ellipsis in the ISA command library 1004. The additional ISA commands may be used in different ways to support accessing fine-grained memory in the two modes described herein. The additional commands are exposed to the operating system 1002 and available for use by programs executing in the operating system 1002. The operating system 1002 optionally includes a JIT compiler 1006, which may be used rather than the extended ISA command set to enable fine-grained memory access in the two modes.

[0052] To provide OS-assisted memory fine-grained memory access in different modes, e.g., the two modes mentioned above of 16B and 32B access, a programmer can compile a program that invokes the normal LD ​​command or the LD_x_MEM and LD_x commands. In other embodiments, instead of new ISA instructions, dual-mode fine-grained access is exposed to the OS, so that the programmer can annotate memory regions that will suffer irregular access. In such embodiments, the programmer requests grain-friendly allocation using a new "malloc" (memory allocation) instruction. In response to such a malloc instruction, the OS tags the page table entry of the corresponding region to indicate the access granularity, i.e., whether the region supports 16B or 32B access. Because processor loads and stores consult the processor's translation lookaside buffer (TLB) or other address translation table before accessing memory, the access granularity tag can be easily retrieved from the address translation of each command. This access granularity tag is preferably communicated as a single bit to the memory controller and indicates whether the associated access has a first width or a second width, for example 16B or 32B.

[0053] A significant limitation of such mechanisms is that the entire allocated memory region must typically have the same access granularity, or that the memory region must receive the same access granularity throughout the entire execution. To address these limitations, some embodiments employ a hybrid approach in which a programmer tags an allocated memory region with an access granularity that can be overridden with a particular access granularity, such as LD vs. LD_X_Mem ISA instructions generated by the compiler. For example, an application may tag a structure as a 32B access granularity region, but the static or JIT compiler 1006 may generate LD_X_Mem ISA instructions that can support 16B accesses. Such a scheme allows the access granularity for a memory region to be dynamically switched from that allocated by the original application programming. As can be seen, the memory 1010 includes fine-grained memory banks that can be operated in either of the two modes as described above, according to instructions from the memory controller 1008, in response to receiving an ISA command or granularity information for the memory region.

[0054] 11 is a flow diagram 1100 of a process for operating fine-grained memory, according to some embodiments. At block 1102, a program is compiled as described with memory access commands that specify an access granularity among two or more granularities, e.g., 16B and 32B. At block 1104, the compiled program is executed by the OS. As shown at block 1106, a just-in-time compiler, such as the JIT compiler 1006, generates fine-grained commands for each specified access granularity based on the fine-grained accesses available in the memory. These commands are interpreted by the memory controller to generate multicast CAS commands to implement memory accesses in which a second access mode is invoked by the JIT compiler.

[0055] The circuits of FIGS. 2, 3, 7, 8, or any portion thereof (such as arbiter 238 or dual mode I / O circuit 708) may be described or represented by a computer-accessible data structure in the form of a database or other data structure that may be read and used directly or indirectly by a program to manufacture an integrated circuit. For example, the data structure may be a behavioral or register-transfer level (RTL) description of the hardware functionality in a high level design language (HDL) such as Verilog or VHDL. The description may be read by a synthesis tool that may synthesize the description to generate a netlist that includes a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware that comprises the integrated circuit. The netlist may then be placed and routed to generate a data set that describes the geometric shapes that are applied to a mask. The mask may then be used in various semiconductor manufacturing processes to manufacture the integrated circuit. Alternatively, the database on the computer accessible storage medium may be a netlist (with or without a synthesis library) or a data set, or Graphic Data System (GDS) II data, if desired.

[0056] Although specific embodiments have been described, various modifications to these embodiments will be apparent to those skilled in the art. For example, memory controller 200 may vary in different embodiments. Memory controller 200 may interface with other types of memory other than DDRx, such as high bandwidth memory (HBM), RAMbus DRAM (RAMbus DRAM, RDRAM), etc. While the illustrated embodiment shows each rank of memory corresponding to a separate DIMM or SIMM, in other embodiments, each module may support multiple ranks. Still other embodiments may include other types of DRAM modules or DRAM not included in a particular module, such as DRAM mounted on a host motherboard. Accordingly, it is intended that the appended claims cover all modifications of the disclosed embodiments that fall within the scope of the disclosed embodiments.

Claims

1. 1. A fine-grained dynamic random access memory (DRAM), comprising: a first memory bank including a memory array divided into a plurality of grains, each grain including a row buffer and an input / output (I / O) circuit; a second memory bank; a dual mode I / O circuit coupled to the I / O circuitry of each grain in the first memory bank, the dual mode I / O circuit operating in a first mode in which commands having a first data width are routed to each grain and executed individually at each grain, and in a second mode in which commands having a second data width different from the first data width are executed in parallel by at least two of the grains; Fine grain DRAM.

2. the dual mode I / O circuitry executes commands in the second mode in response to a multicast column address strobe (CAS) command.

2. The fine grain DRAM of claim 1.

3. the dual mode I / O circuit responds to the multicast CAS command by asserting column address strobes in parallel to selected columns in at least two of the grains.

3. The fine grain DRAM of claim 2.

4. the dual mode I / O circuit includes a multiplexer coupled to a column decoder in the I / O circuit of each grain in the first memory bank; 4. The fine grain DRAM of claim 3.

5. and wherein in the second mode, the dual-mode I / O circuit performs a localized relay of the multicast CAS command to at least one of the at least two of the grains by replacing an initial grain identifier bit in the multicast CAS command with a new value associated with a different grain.

4. The fine grain DRAM of claim 3.

6. the dual mode I / O circuit uses one of a lookup table and an adder to identify the new value; 6. The fine grain DRAM of claim 5.

7. 1. A method of operating a volatile memory, comprising the steps of: sending a column address strobe (CAS) command from a memory controller to a first grain of a first bank on the volatile memory; sending a multicast CAS command to a dual mode I / O circuit on the volatile memory; transmitting, in the volatile memory, a CAS signal to a plurality of grains including the first grain in response to the multicast CAS command; method.

8. reading data having a first data width in response to the CAS command; and reading second data having a second data width different from the first data width in parallel from the plurality of grains in response to the CAS command. The method of claim 7.

9. distributing the multicast CAS command to the plurality of grains in the dual mode I / O circuit; The method of claim 7.

10. and transferring a localized relay of a multicast CAS command to at least one of the plurality of grains by replacing an initial grain identifier bit in the multicast CAS command with a new value associated with a different grain in the dual mode I / O circuit. The method of claim 7.

11. the dual mode I / O circuit uses one of a lookup table and an adder to identify the new value; The method of claim 10.

12. generating the multicast CAS command based on an original memory access request using a just-in-time compiler; The method of claim 7.

13. generating the multicast CAS command based on an original memory access request using a compiler; The method of claim 7.

14. generating the multicast CAS command in a memory controller based on recognizing an instruction set architecture (ISA) command associated with a predetermined data width; The method of claim 7.

15. generating, in a memory controller, the multicast CAS command based on a memory region associated with a memory command processed by the memory controller; The method of claim 7.

16. 1. A data processing system comprising: A plurality of processing units; a data fabric coupled to the plurality of processing units; a memory controller coupled to the data fabric to service memory access requests from the plurality of processing units; a fine-grained dynamic random access memory (DRAM) in communication with the memory controller; The fine grain DRAM comprises: a first memory bank including a memory array divided into a plurality of grains, each grain including a row buffer and an input / output (I / O) circuit; a second memory bank; a dual mode I / O circuit coupled to the I / O circuitry of each grain in the first memory bank, the dual mode I / O circuit operating in a first mode in which commands having a first data width are routed to each grain and executed individually at each grain, and in a second mode in which commands having a second data width different from the first data width are executed in parallel by at least two of the grains; Data processing system.

17. the dual mode I / O circuitry executes commands in the second mode in response to a multicast column address strobe (CAS) command.

17. The data processing system of claim 16.

18. the dual mode I / O circuit responds to the multicast CAS command by asserting column address strobes in parallel to selected columns in at least two of the grains.

20. The data processing system of claim 17.

19. the dual mode I / O circuit includes a multiplexer coupled to a column decoder in the I / O circuit of each grain in the first memory bank; 20. The data processing system of claim 18.

20. and wherein in the second mode, the dual-mode I / O circuit performs a localized relay of the multicast CAS command to at least one of the at least two of the grains by replacing an initial grain identifier bit in the multicast CAS command with a new value associated with a different grain.

20. The data processing system of claim 18.

Images