Method and apparatus for restoring regular access performance in fine-grained dram

Abstract

Fine-grained dynamic random access memory (DRAM) includes a first memory bank, a second memory bank, and dual-mode I / O circuitry. The first memory bank includes a memory array divided into multiple dies, each die including a row buffer and input / output (I / O) circuitry. The dual-mode I / O circuitry is coupled to the I / O circuitry of each die in the first memory bank and operates in a first mode and a second mode. In the first mode, commands having a first data width are routed to each die and implemented individually at each die. In the second mode, commands having a second data width, different from the first data width, are implemented in parallel by at least two of the dies.

Description

Background Technology

[0001] High-bandwidth dynamic random access memory (DRAM) is used by graphics processing units (GPUs) and other throughput-oriented parallel processors. Firstly, through the use of high-frequency off-chip signaling (such as signaling used with graphics double data rate memory), and more recently through integration on the processor die and in the package of wide, high-bandwidth interfaces to the DRAM chip stack (such as those used in high-bandwidth memory (HBM / HBM2) modules), GPU-dedicated DRAM has been primarily optimized for maximizing bandwidth over several generations.

[0002] Many critical workloads in such systems involve irregular, fine-grained memory access. These workloads include graph traversal (widely used in social networking, search, e-commerce, cloud computing, and the gig economy), key-value store access, and high-performance computing (HPC). Next-generation die-based stacked DRAM is important for achieving energy-efficient, high-bandwidth, fine-grained irregular access.

[0003] Fine-grained DRAM (FG-DRAM) enables stacked DRAM memories (such as HBM) to scale memory bandwidth while not exceeding actual thermal design power (TDP) limits. FG-DRAM does this by dividing the DRAM die into smaller, independent cells (called dies), each with a dedicated DQ pin but no command address (CA) pin. For example, a traditional DRAM bank can be divided into two dies, each with half the number of columns compared to the original bank. This results in a significant increase in memory-level parallelism. Additionally, a significant reduction in drive power is achieved.

[0004] However, in such die-based architectures, because each die now has a narrower interface to the DRAM channel, the latency for DRAM atoms to access open rows via column access commands increases proportionally to the number of dies per memory bank. This increase is detrimental to latency-sensitive, regular access workloads. Attached Figure Description

[0005] Figure 1 The accelerated processing unit (APU) and memory system known in the prior art are illustrated in block diagram form;

[0006] Figure 2 The block diagram illustrates the suitability of certain implementation schemes for similar scenarios. Figure 1 The memory controller used in the APU;

[0007] Figure 3 A block diagram comparing two banks of two DRAMs is shown, illustrating the use of a die;

[0008] Figure 4 The arrangement of address lines for fine-grained DRAM memory, interwoven in a die-independent format, is shown in block diagram form.

[0009] Figure 5 The arrangement of address lines in a fine-grained DRAM memory interwoven in a die-strip format is shown in block diagram form.

[0010] Figure 6 illustrates a fine-grained memory according to the prior art in the form of a block diagram;

[0011] Figure 7 A fine-grained storage unit according to some implementation schemes is illustrated in block diagram form;

[0012] Figure 8 A fine-grained storage unit according to some additional implementation schemes is illustrated in block diagram form;

[0013] Figure 9 A flowchart illustrating an operation of a fine-grained memory according to some embodiments is shown;

[0014] Figure 10 A data processing system, including library instructions for operating fine-grained memory, is illustrated in block diagram form according to some embodiments; and

[0015] Figure 11 A flowchart illustrating a process for operating a fine-grained memory according to some embodiments is shown.

[0016] In the following description, the same reference numerals are used in different figures to indicate similar or identical items. Unless otherwise stated, the word “coupled” and its associated verb form include both direct connection and indirect electrical connection by means known in the art, and unless otherwise stated, any description of direct connection also implies alternative embodiments using suitable forms of indirect electrical connection. Detailed Implementation

[0017] Fine-grained dynamic random access memory (DRAM) includes a first memory bank, a second memory bank, and dual-mode I / O circuitry. The first memory bank includes a memory array divided into multiple dies, each die including a row buffer and input / output (I / O) circuitry. The dual-mode I / O circuitry is coupled to the I / O circuitry of each die in the first memory bank and operates in a first mode and a second mode. In the first mode, commands having a first data width are routed to each die and implemented individually at each die. In the second mode, commands having a second data width, different from the first data width, are implemented in parallel by at least two of the dies.

[0018] A method of operating volatile memory includes sending a column address strobe (CAS) command from a memory controller to a first die in a first bank of memory on the volatile memory. The method includes sending a multicast CAS command to dual-mode I / O circuitry on the volatile memory. In response to the multicast CAS command, at the volatile memory, the method includes sending CAS signals to a plurality of dies including the first die.

[0019] The data processing system includes multiple processing units, a data texture coupled to the multiple processing units, a memory controller coupled to the data texture to implement memory access requests from the processing units, and fine-grained dynamic random access memory (DRAM) communicating with the memory controller. The fine-grained DRAM includes a first memory bank, a second memory bank, and dual-mode I / O circuitry. The first memory bank includes a memory array divided into multiple dies, each die including a row buffer and input / output (I / O) circuitry. The dual-mode I / O circuitry is coupled to the I / O circuitry of each die in the first memory bank, and operates in a first mode and a second mode. In the first mode, a command having a first data width is routed to each die and implemented individually at each die. In the second mode, a command having a second data width different from the first data width is implemented in parallel by at least two of the dies.

[0020] Figure 1 An accelerated processing unit (APU) 100 according to some embodiments is illustrated in block diagram form. The APU 100 is implemented as a system-on-a-chip (SoC), which can be part of various host data processing platforms. While an APU is shown in this embodiment, other data processing platforms, such as a central processing unit (CPU) or a graphics processing unit (GPU), can be used. For example, in some embodiments, the fine-grained memory access techniques described herein are embodied in a GPU chip used in a graphics card or other graphics processing module. In other embodiments, a dedicated processor core, such as an intelligent processing unit (IPU), can be used. In this embodiment, the APU 100 typically includes a CPU core complex 110, a graphics core 120, a set of display engines 130, a memory management hub 140, a data texture 150, a set of peripheral controllers 160, a set of peripheral bus controllers 170, a system management unit (SMU) 180, flash memory 205, and a set of FG_DRAM memory controllers 190.

[0021] CPU core complex 110 includes CPU core 112 and CPU core 114. In this example, CPU core complex 110 includes two CPU cores, but in other embodiments, CPU core complex 110 may include any number of CPU cores. Each core in CPU cores 112 and 114 is bidirectionally connected to a system management network (SMN) 145 (which forms a control texture) and a data texture 150, and is able to provide memory access requests to the data texture 150. Each core in CPU cores 112 and 114 may be a monolithic core, or it may be a core complex of two or more monolithic cores sharing certain resources such as cache.

[0022] Each core in the graphics core 120 is a high-performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, and texture blending in a highly integrated and parallel manner. Each graphics core 120 is bidirectionally connected to the SMN 145 and the data texture 150 and can provide memory access requests to the data texture 150. In this regard, the APU 100 can support a unified memory architecture in which the CPU core complex 110 and the graphics core 120 share the same memory space, or a memory architecture in which the CPU core complex 110 and the graphics core 120 share a portion of the memory space, while the graphics core 120 also uses a private graphics memory that the CPU core complex 110 cannot access.

[0023] Display engine 130 renders and rasterizes objects generated by graphics core 120 for display on the monitor. Graphics core 120 and display engine 130 are bidirectionally connected to a common memory management hub 140 for uniform translation to appropriate addresses in memory, and memory management hub 140 is bidirectionally connected to data texture 150 for generating such memory accesses and receiving read data returned from the memory system.

[0024] Data texture 150 includes crossbars for routing memory access requests and memory responses between any memory access agent and memory controller 190. The data texture also includes a system memory map defined by the Basic Input / Output System (BIOS) for determining the destination of memory accesses based on system configuration, and buffers for each virtual connection.

[0025] Peripheral controller 160 includes a USB controller 162 and a Serial Advanced Technology Attachment (SATA) interface controller 164, each of which is bidirectionally connected to system hub 166 and SMN 145. These two controllers are merely examples of peripheral controllers that can be used in APU 100.

[0026] The peripheral bus controller 170 includes a system controller hub 172 and a peripheral controller hub 174, each bidirectionally connected to an input / output (I / O) hub 176 and an SMN 145. The system controller hub 172 is connected to the flash memory 205 via a suitable communication link. The I / O hub 176 is also bidirectionally connected to the system hub 166 and the data texture 150. Therefore, for example, the CPU core can program registers in the USB controller 162, the SATA interface controller 164, the system controller hub 172, or the peripheral controller hub 174 via access routed through the I / O hub 176 from the data texture 150.

[0027] The SMU 180 is a local controller that controls the operation of resources on the APU 100 and synchronizes communication between these resources. The SMU 180 manages the power-on sequence of the various processors on the APU 100 and controls multiple off-chip devices via reset, enable, and other signals. The SMU 180 also manages power for the various processors and other function blocks.

[0028] While a specific SoC implementation is shown, this is not limiting, and other computing platforms can also benefit from the techniques described in this article.

[0029] Figure 2 A block diagram illustrates a suitable approach for applications similar to... Figure 1 The memory controller 200 used in the APU. The memory controller 200 typically includes an interface 212, a memory interface queue 214, a command queue 220, an address generator 222, a content-addressable memory (CAM) 224, replay control logic 231 including a replay queue 230, refresh control logic 232, a timing block 234, a page table 236, an arbitrator 238, an error correction code (ECC) checking circuit 242, an ECC generation block 244, and a data buffer 246.

[0030] Interface 212 has a first bidirectional connection to the data texture via an external bus and has an output. In memory controller 200, this external bus is compatible with Advanced Scalable Interface Version 4 (referred to as AXI4) as specified by ARM Holdings, PLC of Cambridge, England, but may be other types of interfaces in other embodiments. Interface 2)2 translates memory access requests from a first clock domain called the FCLK (or MEMCLK) domain to a second clock domain called the UCLK domain within memory controller 200. Similarly, memory interface queue 214 provides memory access from the UCLK domain to the DFICLK domain associated with the DFI interface.

[0031] Address generator 222 decodes the address of a memory access request received from the data texture via the AXI4 bus. The memory access request includes an access address in a normalized format within the physical address space. Address generator 222 translates the normalized address into a format that can be used to address the actual memory devices in the memory system and to efficiently schedule related accesses. This format includes region identifiers that associate the memory access request with specific memory column, row address, column address, bank address, and bank group. At startup, the system BIOS queries the memory devices in the memory system to determine their size and configuration and programs a set of configuration registers associated with address generator 222. Address generator 222 uses the configuration stored in the configuration registers to translate the normalized address into the appropriate format. Command queue 220 is a queue of memory access requests received from memory access agents in APU 100, such as CPU cores 112 and 114 and graphics core 120. Command queue 220 stores the address fields decoded by address generator 222, as well as other address information that allows arbitrator 238 to efficiently select memory accesses, including access type and quality of service (QoS) identifiers. CAM 224 includes information for implementing sorting rules such as Write-After-Write (WAW) and Read-After-Write (RAW) sorting rules. Command queue 220 is a stacked command queue comprising multiple stacks of entries, each stack containing multiple command entries. In this embodiment, four entries each contain stacks of 32 entries, as further described below.

[0032] Error Correction Code (ECC) generation block 244 determines the ECC of the write data to be sent to memory. This ECC data is then added to the write data in data buffer 246. ECC checking circuit 242 checks the received ECC against the incoming ECC.

[0033] Replay queue 230 is a temporary queue for storing selected memory accesses chosen by arbitrator 238, which are awaiting responses such as address and command parity responses. Replay control logic 231 accesses ECC checking circuit 242 to determine whether the returned ECC is correct or indicates an error. Replay control logic 231 initiates and controls a replay sequence in which accesses are replayed if a parity error or ECC error occurs in one of the cycles. The replayed commands are placed in memory interface queue 214.

[0034] Refresh control logic 232 includes a state machine for various power-down, refresh, and termination resistor (ZQ) calibration cycles, which are generated separately from normal read and write memory access requests received from the memory access agent. For example, if a memory column is in pre-charge power-down, that memory column must be periodically woken up to run a refresh cycle. Refresh control logic 232 periodically and in response to specified conditions generates refresh commands to prevent data errors caused by charge leakage from the storage capacitors of memory cells in the DRAM chip. Refresh control logic 232 includes an activation counter 248, which in this embodiment has a counter for each memory region that counts the number of roll-through activation commands sent to the memory region via the memory channel. A memory region is a memory bank in some embodiments and a memory sub-bank in other embodiments, as discussed further below. Furthermore, refresh control logic 232 periodically calibrates ZQ to prevent mismatch of on-chip termination resistors due to thermal variations in the system.

[0035] Arbitrator 238 is bidirectionally connected to command queue 220 and is the core of memory controller 200, performing intelligent scheduling of accesses to improve memory bus utilization. In this embodiment, arbitrator 238 includes a die address translation block 240 for translating commands and addresses for fine-grained memory accesses in multiple modes, as further described below. In some embodiments, such functionality may be performed by address generator 222. Arbitrator 238 uses timing block 234 to implement correct timing relationships by determining whether certain accesses in command queue 220 are eligible to be issued based on DRAM timing parameters. For example, each DRAM has a minimum specified time between activation commands, referred to as 't'. RC Timing block 234 maintains a set of counters that determine eligibility based on the timing parameters and other timing parameters specified in the JEDEC specification, and this timing block is bidirectionally connected to replay queue 230. Page table 236 maintains status information about active pages in each bank and column of memory channels of arbitrator 238, and is bidirectionally connected to replay queue 230. Arbitrator 238 includes a single command input for each entry of stack in command queue 220, and selects commands from it to schedule dispatch to DRAM channels via memory interface queue 214.

[0036] In response to receiving a write memory access request from interface 212, ECC generation block 244 calculates the ECC based on the write data. Data buffer 246 stores the write data and ECC for the received memory access request. When arbitrator 238 selects a corresponding write access for dispatch to the memory channel, the data buffer outputs the combined write data / ECC to memory interface queue 214.

[0037] The memory controller 200 includes circuitry that allows it to select memory accesses for assignment to associated memory channels. To make a desired arbitration decision, address generator 222 decodes address information into pre-decoded information, including the memory column, row, column, bank, and bank groups in the memory system, and command queue 220 stores the pre-decoded information. A configuration register (not shown) stores configuration information to determine how address generator 222 decodes the received address information. Arbitrator 238 uses the decoded address information, timing eligibility information indicated by timing block 234, and active page information indicated by page table 236 to efficiently schedule memory accesses while adhering to other criteria such as Quality of Service (QoS) requirements. For example, arbitrator 238 implements priority access to open pages to avoid the overhead of precharge and activation commands required to change memory pages, and hides overhead access to one bank by interleaving overhead access to one bank with read and write access to another bank. Especially during normal operation, the arbitrator 238 typically keeps pages open in different memory banks until these pages need to be precharged, and then selects a different page.

[0038] Figure 3 Block diagram 300 shows two memory banks 302 and 304 of two DRAMs, illustrating the use of dies. DRAM bank 302 is constructed without dies, while DRAM bank 304 includes two dies 306 and 308. DRAM bank 304 is part of a larger DRAM integrated circuit that includes multiple DRAM banks. DRAM bank 304 typically includes a memory array divided into multiple dies (two in this case), each die including a row buffer and input / output (I / O) circuitry.

[0039] The conventional amorphous architecture of DRAM bank 302 performs 32-byte (32B) memory accesses via 32 data lines DQ[0:31], accessing in 8 bursts or "beats" on the pseudo-channel (pCH) allocated to DRAM bank 302. Issuing column commands to open rows in the bank provides 32 bytes (32B) of data in 8 bursts, thus providing twice the data read speed compared to the two-die architecture of DRAM bank 304, as the pins are not partitioned. Consequently, in the amorphous architecture, reading a 64B cache line from an open row takes 2*8 = 16 bursts plus an additional column-to-column length delay (tCCDL) minus the burst duration (tBURST) on the data bus.

[0040] When using a fine-grained architecture without the techniques described herein, the DRAM bank 304 is accessed in 16-bit access via two designated data line sets DQ[0:15] and DQ[16:31] of the pCH. In this exemplary arrangement with 16 DQ pins having a pseudo-channel dedicated to the die, the open rows in the die respond to column commands by providing 16-bit data in 8 bursts. Therefore, reading a 64-bit cache line from the open rows takes 4*8=32 bursts, plus an additional 3(tCCDL-tBURST) cycles of overhead. Thus, for a single 32-bit access, the die architecture results in a 33% latency penalty.

[0041] Figure 4 The arrangement of address lines for fine-grained DRAM memory, interleaved in a die-independent format, is illustrated graphically. In the depicted arrangement, address line numbers are shown at the top, with address lines marked below each address line number. "RO" represents row, "CO" represents column, "GR" represents die number, "BK" represents bank number, "BG" represents bank group number, "PCH" represents pseudo-channel, and "X" indicates an unassigned address line. In this die-independent address interleaving, striping is not performed to enable 16-bit access for irregular throughput, and the die address bit at address line 12 is used similarly to the bank address bits.

[0042] Figure 5 The arrangement of address lines in a fine-grained DRAM memory interleaved in a die-strip format is illustrated graphically. The depicted implementation deploys modified address interleaving at the memory controller to achieve a stripe 32B placement of dies with die-strip address interleaving. Data 32B is divided into two associated dies, each 16B in size. A GR address bit is located at the least significant address line (line 4), meaning that adjacent dies are addressed by this bit. It should be noted that this method can be extended to support an increased number of dies per memory bank, such as four, eight, or sixteen dies (e.g., theoretically limited only by the number of columns in the memory bank).

[0043] However, due to the tCCDL constraint between column commands to the same physical storage, using only one... Figure 5 The die-strip address interleaving still suffers from an unacceptable additional 75% overhead. In some implementations, this overhead can be reduced to tCCDS overhead by decoupling "buddy" dies (dies that operate with neighboring addresses) into different memory banks. However, even in such implementations, the overhead remains non-negligible. Furthermore, such decoupling of buddy dies into different memory banks imposes additional scheduling and data response reorganization complexity on the memory controller to avoid asynchronous latency between the two dies.

[0044] Figure 6 illustrates a fine-grained memory 600 according to the prior art in block diagram form, however Figure 7 A fine-grained storage 700 according to some implementation schemes is illustrated in block diagram form.

[0045] The memory bank 600 includes two dies labeled "Die 0" and "Die 1", a row decoder 602, two column decoders 604 and 606, a die selection circuit 608 labeled "Gr.Sel", a command bus 620, and an address bus 622. In operation, the memory bank 600 is accessed using commands, such as the depicted column address strobe (CAS) command sent from the memory controller via the command bus 620. As shown in the figure, according to... Figure 4 The address scheme uses address bus 622 to provide column address bits (CO) to column decoders 604 and 606, and to provide die bits (GR) to die selector circuit 608. Memory access is one die width, which is 16 bytes in this example.

[0046] See Figure 7 The memory bank 700 includes two dies labeled "Die 0" and "Die 1", and I / O circuitry including a row decoder 702, two column decoders 704 and 706, a row buffer (not shown separately) in each die, a dual-mode input / output (I / O) circuitry 708 labeled "mCAS support", and a command bus and address bus 722 labeled 720. In this specific implementation, the dual-mode I / O circuitry 708 is connected to the I / O circuitry of each die in the memory bank.

[0047] The dual-mode I / O circuit 708 includes a first input labeled "mCAS Enable", a second input labeled "Die Bit", a first output labeled "Die 0 Enable", and a second output labeled "Die 1 Enable". The first input receives a multicast CAS enable signal from the memory controller via the command bus 720, and the second input receives the die address bit (GR). Figure 5 The first output is connected to column decoder 704, and the second output is connected to column decoder 706.

[0048] The dual-mode I / O circuit 708 typically operates in a first mode and a second mode. In the first mode, commands with a first data width (16B) are routed to each die and implemented individually at each die. In the second mode, commands with a second data width (32B) different from the first data width are implemented in parallel by at least two of the dies. This arrangement overcomes the overhead imposed by the tCCDL constraints discussed above by employing multicast column address strobe (mCAS) commands for two or more partner dies (which in this example are the two dies of memory bank 700). Except for the die identifier bits, the address bits constituting the mCAS command are identical for both dies. Thus, the depicted specific implementation enhances the shared command and address (CA) path / logic by leveraging support for mCAS commands.

[0049] In the depicted two-die example, the dual-mode I / O circuit 708 is implemented using a multiplexer enhanced by an mCAS enable signal, but other implementations employ different logic specificities. The dual-mode I / O circuit 708 is shown in more detail on the right side of the figure, and in this specific embodiment includes a multiplexer 710 and two OR gates 712. The multiplexer 710 has an input that receives die address bits and two outputs connected to corresponding inputs of the OR gates 712. Each OR gate 712 has a first input connected to an output of the multiplexer 710 and a second input that receives a multicast CAS enable signal.

[0050] In operation, when the multicast CAS enable signal is high to signal that the current CAS command is a multicast CAS command, the output of OR gate 712 goes high, thereby enabling the multicast CAS command by causing the CAS to be asserted for a selected column in both dies of memory bank 700. The dies then respond to the CAS in parallel to provide a higher data width. In this mode, both dies receive the mCAS command, and both respond in parallel to reading 16B of data from the corresponding address indicated on the address bus. When the multicast CAS enable signal is low, only one OR gate 712 has a high output at a time, enabling only one die at a time to receive the CAS command, thus providing normal fine-grained operation.

[0051] While the memory bank 700 in this specific embodiment has two dies, higher die counts are provided in some embodiments, including a multi-stage multiplexer tree in the dual-mode I / O circuit 708. For example, in an exemplary embodiment with eight dies in the memory bank 700, each die is capable of providing fine-grained memory access with a width of 4B in a first mode, while all eight dies operating in parallel in a second mode provide access with a width of 32B. The dual-mode I / O circuit 708 in such embodiments includes a three-stage multiplexer tree that receives three die bits. In the first mode, the multiplexer tree routes CAS commands to the appropriate die being addressed and activates only a single decoder out of the eight column decoders. In the second mode, the dual-mode I / O circuit responds to multicast CAS commands by enabling all eight column decoders, thereby asserting the CAS command at each die.

[0052] Figure 8 A fine-grained memory bank 800 according to some additional embodiments is illustrated in block diagram form. The memory bank 800 includes two dies labeled "Die 0" and "Die 1", and I / O circuitry including a row decoder 802, two column decoders 804 and 806, a row buffer (not shown separately) in each die, a die selection circuit 808 labeled "Gr.Sel", a multicast CAS relay circuit 810 labeled "mCAS relay", a command bus 820, and an address bus 822.

[0053] In this implementation, the dual-mode I / O circuitry is represented as a multicast CAS relay circuit 810, rather than as a specific implementation of a multiplexer tree, such as... Figure 7 The multiplexer tree implementation, when there are a large number of dies per memory bank (e.g., the number of dies "G", such as 8 or 16), results in an increase in critical path command latency due to the additional depth of the multiplexer tree implementation. Instead of using a multiplexer tree, the depicted implementation performs localization and asynchronous repetition (relay) of the CAS signal using a multicast CAS relay circuit 810. The multicast CAS relay circuit 810 is co-located with the column decoder 804 of die 0 and includes a first input that receives the mCAS enable signal and an output connected to the column decoder of die 1. In implementations with more than two dies, the multicast CAS relay circuit 810 is present in the I / O circuitry for multiple dies to forward the mCAS to other dies. The multicast CAS relay circuit 810 has lower capacitance than the multiplexer tree implementation due to its shorter electrical conductors, and therefore does not increase command latency much when there are many dies.

[0054] In operation, memory bank 800 is accessed using commands, such as the depicted column address strobe (CAS) command sent from the memory controller via the command bus. As shown in the figure, according to... Figure 4 The address scheme uses column address bits (CO, Figure 4 ) is carried into column decoders 804 and 806, and the die bits (GR, Figure 4 The memory access is carried to the die selector circuit 808. In the first mode, the memory access is the width of one die, which is 16 bytes in this example. In the second mode, the memory access is the width of two or more dies (two in this example, totaling 32 bytes). The second mode is activated by an mCAS enable signal fed from the memory controller to the multicast CAS relay circuit 810. In the second mode, the multicast CAS relay circuit 810 executes in the second mode to relay the local relay of the multicast CAS command to another die among at least two dies in that die by replacing the initial die identifier value in the multicast CAS command with a new value identifying the other die. In some embodiments, the multicast CAS relay circuit 810 uses a lookup table or adder to identify the new value.

[0055] Although about Figure 7 and Figure 8 Two different specific implementations for dual-mode I / O circuitry are described, but other suitable digital logic schemes can be implemented in memory to perform the operational steps in the first and second modes.

[0056] Figure 9 A flowchart 900 is shown for a process for operating fine-grained memory according to some embodiments. In this specific embodiment, the process employs three variations of the memory load (read) command as listed on the right, including an "LD" command that makes a 32-B request to main memory in the event of a cache miss, an "LD_x" command that specifies an irregular load size "x" that can be obtained from an on-chip register instead of main memory, and an "LD_x_MEM" command that specifies an irregular load of size x that can be obtained from main memory via a single 16-B request.

[0057] As shown in box 902, the process includes marking the selected LD memory access request as an LD_x_MEM request if it is an irregular request that must result in a 16B fetch from main memory. The marking at box 902 can be performed by the programmer who created the original software in which the memory access command occurs, or by a compiler or just-in-time compiler that interprets the programming instructions used to specify the memory system.

[0058] At block 904, the memory controller receives a memory access request associated with a marked memory access request, such as when a program or library executes the instruction marked at block 902. The request may have a first data width, such as 16B as shown, or a second data width, such as 32B as shown. At block 906, the process determines whether the request has a first data width, such as 16B. If so, the process proceeds to block 908, where die-independent address interleaving is applied to the request to implement it with a fine-grained memory access of 16B width. The memory controller then issues a CAS command to the 16B-width memory die to implement the command. As discussed above, the request is implemented in a first mode using access to a single die of memory in the corresponding memory bank. When an activation is issued for the 16B CAS at block 910, the memory controller receives response data from memory in eight bursts of 16B.

[0059] Referring again to box 906, if the request is not a 16B request, the process proceeds to boxes 912 through 914, where the request is implemented using dual-mode I / O circuitry operating in the second mode as described above. At box 912, the 32B request is split into two 16B sub-requests, which are determined according to, for example... Figure 5 The dies shown are arranged in a striped configuration and are related to each other. At block 914, address interleaving for this die-strip scheme is applied to two 16-bit commands, and a 32-bit multicast CAS command is issued from the memory controller to implement the two commands from the memory. In response to the multicast CAS command, the memory fulfills the request by having column address gating for selected columns in at least two dies asserted in parallel. For example, 708 ( Figure 7 808 dual-mode I / O circuit or multicast relay circuit Figure 8 The system operates in a second mode to allow multicast CAS commands to be asserted at multiple memory dies. The resulting data is returned to the memory controller in two parallel bursts of 16 bytes each, as shown in box 916. The memory controller can also perform a check on the received data to provide a response to the original memory request for data in the appropriate order.

[0060] Figure 10 A data processing system 1000, comprising library instructions for operating fine-grained memory, is illustrated in block diagram form according to some embodiments. The data processing system 1000 includes an operating system (OS) 1002, a memory controller 1008, and a memory 1010. The operating system 1002 executes on one or more processors of the data processing system and includes an instruction set architecture (ISA) command library 1004 and a just-in-time (JIT) compiler 1006.

[0061] In some specific implementations, in addition to the standard LD command and other commonly used commands in the ISA library as indicated by the ellipses in the ISA command library 1004, the extended ISA command library 1004 also includes the load commands LD_x and LD_x_MEM, as mentioned above. Figure 9 As described herein, additional ISA commands can be used in different ways to support fine-grained memory access in two modes as described herein. The additional commands are exposed to the operating system 1002 and can be used by programs executed within the operating system 1002. The operating system 1002 optionally includes a JIT compiler 1006 instead of the extended ISA command set, which can be employed to implement fine-grained memory access in both modes.

[0062] To provide fine-grained memory access to OS-assisted memory in different modes (e.g., the two modes described above for 16B and 32B access), programmers can compile programs that call regular LD commands or LD_x_MEM and LD_x commands. In other implementations, instead of new ISA instructions, dual-mode fine-grained access is exposed to the OS, allowing programmers to annotate memory regions that will result in irregular accesses. In such implementations, programmers use new "malloc" (memory allocation) instructions to request favorable allocations for the die. In response to such malloc instructions, the OS marks the page table entries for the corresponding region to indicate the access granularity, i.e., whether the region supports 16B or 32B access. Since the processor consults the processor's Translation Lookaside Buffer (TLB) or other address translation tables before loading and storing memory accesses, the access granularity flag can be easily retrieved from the address translations used for each command. This access granularity flag is preferably conveyed to the memory controller as a single bit, indicating whether the associated access has a first width or a second width, such as 16B or 32B.

[0063] A significant limitation of such mechanisms is that the entire allocated memory region typically needs to have the same access granularity, or the memory region needs to generate the same access granularity throughout execution. To address these limitations, some implementations employ a hybrid approach, where the programmer labels the allocated memory region with an access granularity that can be overridden by a specific access granularity (such as the LD_X_MemISA instruction generated by the compiler). For example, an application might label the structure as a 32-bit access granularity region; however, a static compiler or JIT compiler 1006 could create an LD_X_MemISA instruction that supports 16-bit access. Such schemes allow the access granularity of memory regions to be dynamically switched from the access granularity allocated programmatically by the original application. As will be understood, memory 1010 includes fine-grained storage that can operate in either of the two modes described above, in response to receiving an ISA command or granularity information of the memory region, according to instructions from memory controller 1008.

[0064] Figure 11 A flowchart 1100 illustrating a process for operating fine-grained memory according to some embodiments is shown. At block 1102, as described, a program is compiled with memory access commands specifying access granularities from more than one granularity (e.g., 16B and 32B). At block 1104, the compiled program is executed by the OS. As shown at block 1106, based on the fine-grained accesses available at the memory location, a just-in-time compiler (such as JIT compiler 1006) creates fine-grained commands for the corresponding specified access granularities. These commands are interpreted by the memory controller to create multicast CAS commands for implementing memory access, where a second access mode is invoked by the JIT compiler.

[0065] Figure 2 , Figure 3 , Figure 7 , Figure 8The circuitry or any part thereof (such as arbiter 238 and dual-mode I / O circuitry 708) may be described or represented by a computer-accessible data structure in the form of a database or other data structure that can be read by a program and is used directly or indirectly for manufacturing integrated circuits. For example, the data structure may be a behavioral-level description or register-transfer-level (RTL) description of hardware functionality in a high-level design language (HDL) such as Verilog or VHDL. The description may be read by a synthesis tool that synthesizes the description to produce a netlist including a list of gates from a synthesis library. The netlist includes gate sets that also represent the functionality of the hardware comprising the integrated circuit. The netlist can then be placed and routed to produce a dataset describing the geometry to be applied to a mask. The mask can then be used in various semiconductor manufacturing steps to produce the integrated circuit. Alternatively, the database on a computer-accessible storage medium may be a netlist (with or without a synthesis library), a dataset (as needed), or Graphical Data System (GDS) II data.

[0066] While specific embodiments have been described, various modifications to these embodiments will be apparent to those skilled in the art. For example, the internal architecture of the memory controller 200 may vary in different embodiments. The memory controller 200 may interface with other types of memory besides DDRx, such as high-bandwidth memory (HBM), RAMbus DRAM (RDRAM), etc. While the illustrated embodiments show each memory storage column corresponding to a single DIMM or SIMM, in other embodiments, each module may support multiple storage columns. Still other embodiments may include other types of DRAM modules or DRAM not included in a particular module, such as DRAM mounted to a motherboard. Therefore, the appended claims are intended to cover all modifications of the disclosed embodiments that fall within the scope of the disclosed embodiments.

Claims

1. A fine-grained dynamic random access memory, the fine-grained dynamic random access memory comprising: A first memory bank includes a memory array divided into multiple dies, each die including a row buffer and input / output I / O circuitry. Second storage; and A dual-mode I / O circuit is coupled to the I / O circuit of each die in the first memory bank. The dual-mode I / O circuit operates in a first mode and a second mode. In the first mode, a command having a first data width is routed to each die and implemented individually at each die. In the second mode, a command having a second data width different from the first data width is implemented in parallel by at least two of the dies.

2. The fine-grained dynamic random access memory according to claim 1, wherein the dual-mode I / O circuitry responds to a multicast column address strobe command to implement the command in the second mode.

3. The fine-grained dynamic random access memory of claim 2, wherein the dual-mode I / O circuitry responds to the multicast column address strobing command by asserting column address strobing in parallel for selected columns of at least two of the dies.

4. The fine-grained dynamic random access memory according to claim 3, wherein the dual-mode I / O circuitry includes a multiplexer coupled to a column decoder in the I / O circuitry of each die in the first memory bank.

5. The fine-grained dynamic random access memory of claim 3, wherein the dual-mode I / O circuitry executes in the second mode to forward a local relay of the multicast column address strobe command to at least one of the at least two of the at least two in the at least one ...

6. The fine-grained dynamic random access memory of claim 5, wherein the dual-mode I / O circuitry uses either a lookup table or an adder to identify the new value.

7. A method of operating volatile memory, the method comprising: Send a column address strobe CAS command from the memory controller to the first die in the first memory bank on the volatile memory; Send a multicast CAS command to the dual-mode I / O circuit on the volatile memory; as well as In response to the multicast CAS command, a CAS signal is sent at the volatile memory to a plurality of dies including the first die.

8. The method according to claim 7, further comprising: In response to the column address strobe CAS command, data is read with a first data width, and in response to the CAS signal, second data is read in parallel from the plurality of dies with a second data width different from the first data width.

9. The method according to claim 7, further comprising: At the dual-mode I / O circuit, the multicast CAS command is assigned to the plurality of chips.

10. The method according to claim 7, further comprising: At the dual-mode I / O circuit, the multicast CAS command is locally relayed to at least one of the plurality of chips by replacing the initial chip identifier bit in the multicast CAS command with a new value associated with a different chip.

11. The method of claim 10, wherein the dual-mode I / O circuit uses one of a lookup table and an adder to identify the new value.

12. The method according to claim 7, further comprising: The multicast CAS command is created using a just-in-time compiler based on the original memory access request.

13. The method according to claim 7, further comprising: The multicast CAS command is created using a compiler based on the original memory access request.

14. The method according to claim 7, further comprising: At the memory controller, the multicast CAS command is created based on the instruction set architecture command that is identified and associated with a predetermined data width.

15. The method according to claim 7, further comprising: At the memory controller, the multicast CAS command is created based on the memory region associated with the memory command processed by the memory controller.

Images