Leverage Processing-in-Memory (PIM) resources to accelerate non-PIM instructions executed on the host

Abstract

Leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host is disclosed. A memory controller identifies a first write instruction to write first data to a first memory location, the first write instruction being not a PIM instruction. The memory controller writes the first data to a first PIM register. Opportunistically, the memory controller moves the first data from the first PIM register to the first memory location. The memory controller also identifies a first memory location associated with a first read instruction, the first read instruction being not a PIM instruction. The memory controller identifies a PIM register associated with the first memory location. The memory controller reads the first data from the PIM register in response to the first read instruction.

Description

[Background technology]

[0001] A computing system often includes several processors that can fetch and execute instructions and store the results of the executed instructions in suitable locations. A processor (e.g., a central processing unit (CPU) or a graphics processing unit (GPU)) may include several functional units, such as arithmetic logic unit (ALU) circuitry, floating point unit (FPU) circuitry, and / or combinational logic blocks, that can be used to execute instructions by performing arithmetic operations on data. For example, the functional unit circuitry can be used to perform arithmetic operations such as addition, subtraction, multiplication, and / or division on operands. Typically, the processor is coupled to an external memory, and data is accessed via a bus or interconnect between the processor and the memory to execute a set of instructions. To reduce the number of accesses to fetch or store data in memory, a computing system may use a cache hierarchy that temporarily stores recently accessed or modified data for use by a processor or group of processors. However, processing performance can be further improved by offloading certain operations to memory-centric execution units, where processing resources are implemented within and / or near memory, such that data processing is performed closer to the memory locations that store the data, rather than bringing the data closer to the processor. Memory-centric execution devices can save time and also save power by reducing external communication (i.e., from the processor to the memory).

[0002] One platform for memory-centric processing is the processing-in-memory (PIM) architecture. The PIM architecture supports the execution of memory-intensive tasks while reducing the workload on the host processor. For example, a processor core may implement an instruction set architecture (ISA) that includes special instructions, such as PIM instructions, to offload operations to a memory-centric processor, such as a PIM unit. When a processor core executes a PIM instruction, the operation indicated in the PIM instruction is not executed on the processor core. Instead, the PIM instruction is offloaded to a PIM unit in a PIM-enabled memory device for execution. Upon completion of the PIM instruction in the processor core, the memory controller issues the PIM instruction, which includes the PIM command, operand values, memory addresses, and other metadata. Upon receiving the PIM instruction, the PIM unit executes the specified PIM operation. To execute the PIM operation, the PIM unit includes an arithmetic logic unit and a register file for holding operand values ​​received from the core and intermediate values ​​of the PIM operation. In one example, one PIM unit is provided for each memory bank.

[0003] An application or compiled code may contain PIM code to utilize PIM resources for memory intensive tasks. However, not all parts of an application are suitable for acceleration using PIM. In addition, there are applications that do not have code suitable for PIM acceleration. When a processor core is not issuing PIM instructions to a memory bank, the PIM resources are idle and the PIM registers in the PIM register file are unused. [Brief description of the drawings]

[0004] [Figure 1]FIG. 1 is a block diagram of an example system for leveraging processing-in-memory (PIM) resources to accelerate non-PIM instructions executing on a host, in accordance with an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a block diagram of an example memory system that supports leveraging PIM resources to expedite non-PIM instructions executed on a host, in accordance with an embodiment of the present disclosure. [Diagram 3] FIG. 2 is a flow diagram illustrating an example method for leveraging PIM resources to expedite non-PIM instructions executing on a host, according to an embodiment of the present disclosure. [Figure 4] FIG. 11 is a flow diagram illustrating another example method for leveraging PIM resources to expedite non-PIM instructions executing on a host, according to an embodiment of the present disclosure. [Diagram 5] FIG. 11 is a flow diagram illustrating another example method for leveraging PIM resources to expedite non-PIM instructions executing on a host, according to an embodiment of the present disclosure. [Figure 6] FIG. 11 is a flow diagram illustrating another example method for leveraging PIM resources to expedite non-PIM instructions executing on a host, according to an embodiment of the present disclosure. [Figure 7] FIG. 11 is a flow diagram illustrating another example method for leveraging PIM resources to expedite non-PIM instructions executing on a host, according to an embodiment of the present disclosure. [Figure 8] FIG. 11 is a flow diagram illustrating another example method for leveraging PIM resources to expedite non-PIM instructions executing on a host, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Embodiments according to the present disclosure relate to leveraging processing-in-memory (PIM) resources, such as PIM registers, to expedite non-PIM memory instructions. In various embodiments, the PIM registers are used to buffer data for non-PIM read and write instructions so that data can be opportunistically transferred between the PIM registers and the memory array, for example, when the bank is idle or when the memory row is already open. In some embodiments, the PIM registers are used as write buffers to increase the speed at which non-PIM write instructions are drained from the memory controller write queue. Writing data to a PIM register is faster than reading from the memory because it does not require opening and closing of the memory row. Data written to a PIM register may be for the memory array, for example, when the memory bank is idle. In other embodiments, the PIM registers are used as read buffers (e.g., staging buffers, early fetch buffers, prefetch buffers and / or memory-side caches) to reduce read latency. Data targeted by a non-PIM read instruction can be read into a PIM register prior to picking the non-PIM read instruction for dispatch by the memory controller. When the non-PIM read instruction is ready to be picked by the memory controller, the data is read from the PIM register, which is faster since there is no need to open and close memory rows. Additional features and advantages of embodiments will be recognized by those skilled in the art in view of the following disclosure.

[0006] One embodiment is a method for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host. The method includes identifying a first write instruction for writing first data to a first memory location, the first write instruction being a non-processing-in-memory (PIM) instruction. The method also includes writing the first data to a first PIM register. The method further includes moving the first data from the first PIM register to the first memory location. In one variation, the first data is moved to the first memory location during a bank idle period. In another variation, the first data is written to the first PIM register during a bank refresh period. In yet another variation, the first data is moved to the first memory location while the memory channel is in a read mode. In an additional variation, the first data is written to the first PIM register by dispatching a PIM write instruction, and the first data is stored to the first memory location by dispatching a PIM store instruction.

[0007] In some variations, writing the first data to the first PIM register includes updating a PIM register mapping table to include a first entry associating the first PIM register with the first memory location. In some variations, the method further includes identifying a second entry in the PIM register mapping table that associates the second PIM register with the second memory location. In these variations, the first memory location and the second memory location are included in one memory row. In these variations, the method further includes moving the second data in the second PIM register to the second memory location during a row activation phase in which the first data is moved to the first memory location.

[0008] Another embodiment relates to another method of leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host. The method includes identifying a first memory location associated with a first read instruction, where the first read instruction is not a PIM instruction. The method also includes identifying that a PIM register is associated with the first memory location. The method further includes reading the first data from the PIM register in response to the first read instruction. In some variations, an entry in a PIM register mapping table associates the first PIM register with the first memory location. In some variations, the first data is read from the PIM register by dispatching a PIM read instruction.

[0009] In some variations, the method further includes loading the first data from the first memory location into the PIM register. In these variations, the method also includes updating a PIM register mapping table to include an entry associating the PIM register with the first memory location. In some variations, the first data is loaded into the PIM register by dispatching a PIM load instruction. In some variations, the first data is loaded into the PIM register while the memory channel is in a write mode and the first data is read from the PIM register while the memory channel is in a read mode. In some variations, the first data is speculatively loaded into the PIM register.

[0010] In some variations, the method further includes reading the second data from the second memory location, where the first memory location and the second memory location correspond to consecutive columns in the memory row. In these variations, the first data is loaded into the PIM register from the first memory location while the second data is read from the second memory location. In some variations, a single command is dispatched to read the second data from the second memory location and load the first data from the first memory location. In some variations, a single command is dispatched to read the first data from the PIM register and close the memory row.

[0011] Yet another embodiment relates to an apparatus for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host. The apparatus includes a processor and a memory controller coupled to the processor. The memory controller includes logic configured to identify a first write instruction for writing first data to a first memory location, the first write instruction being not a processing-in-memory (PIM) instruction. The logic is also configured to write the first data to a first PIM register. The logic is further configured to move the first data from the first PIM register to the first memory location. In one variation, the first data is moved to the first memory location during a bank idle period. In another variation, the first data is written to the first PIM register during a bank refresh period. In yet another variation, the first data is moved to the first memory location while the memory channel is in a read mode. In an additional variation, the first data is written to a first PIM register by dispatching a PIM write instruction, and the first data is stored to a first memory location by dispatching a PIM store instruction.

[0012] In some variations, writing the first data to the first PIM register includes updating a PIM register mapping table to include a first entry associating the first PIM register with the first memory location. In some variations, the logic is further configured to identify a second entry in the PIM register mapping table that associates the second PIM register with the second memory location. In these variations, the first memory location and the second memory location are included in a memory row. In these variations, the logic is further configured to move the second data in the second PIM register to the second memory location during a row activation phase in which the first data is moved to the first memory location.

[0013] Yet another embodiment relates to another apparatus for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host. The apparatus includes a processor and a memory controller coupled to the processor. The memory controller includes logic configured to identify a first memory location associated with a first read instruction, the first read instruction being not a PIM instruction. The logic is also configured to identify that a PIM register is associated with the first memory location. The logic is further configured to read the first data from the PIM register in response to the first read instruction. In some variations, an entry in a PIM register mapping table associates the first PIM register with the first memory location. In some variations, the first data is read from the PIM register by dispatching a PIM read instruction.

[0014] In some variations, the logic is further configured to load the first data from the first memory location into the PIM register. In these variations, the logic is further configured to update the PIM register mapping table to include an entry associating the PIM register with the first memory location. In some variations, the first data is loaded into the PIM register by dispatching a PIM load instruction. In some variations, the first data is loaded into the PIM register while the memory channel is in a write mode and the first data is read from the PIM register while the memory channel is in a read mode. In some variations, the first data is speculatively loaded into the PIM register.

[0015] In some variations, the logic is further configured to read the second data from the second memory location, where the first memory location and the second memory location correspond to consecutive columns in one memory row. In these variations, the first data is loaded into the PIM register from the first memory location while the second data is read from the second memory location. In some variations, a single command is dispatched to read the second data from the second memory location and load the first data from the first memory location. In some variations, a single command is dispatched to read the first data from the PIM register and close the memory row.

[0016] An embodiment according to the present disclosure will be described in more detail beginning with FIG. 1. Throughout the specification and drawings, like numerals refer to like components. FIG. 1 illustrates a block diagram of an exemplary system 100 for leveraging processing-in-memory (PIM) resources to accelerate non-PIM instructions executed on a host according to the present disclosure. The exemplary system 100 of FIG. 1 includes a host device 170 having a processor 132 including one or more processor cores 104. Although four processor cores are shown in FIG. 1, it should be understood that the host device 170 can include more or fewer processor cores than shown. In various examples, the processor core 104 is a CPU core, a GPU core, or other cores known to those skilled in the art. The processor core 104 issues memory instructions or commands to transfer data between the processor 132 and the memory device 180, for example, by reading data stored in the memory device into a processor cache or writing data in the processor cache to the memory device. These operations are typically performed through conventional load and store instructions dispatched by memory controller 140 to memory devices 180 through a last level cache 160 shared by cores 104. In some examples, host device 170 includes one memory controller 140 per memory channel 188, where a memory channel corresponds to one or more banks 184, 186, or some other memory partition. Memory controller 140 maintains one or more dispatch queues, such as write queue 110 and read queue 112, for queuing memory instructions to be dispatched to a memory channel or other memory partition. Command bus 144 provides an interface between memory controller 140 and memory devices for dispatching memory instructions and commands. Data bus 146 provides an interface for transferring data between memory devices 180 and host device 170.

[0017] In the example of FIG. 1, the memory device 180 is a PIM-enabled memory device including at least one PIM unit 150 for executing PIM instructions dispatched by the processor core 104. For example, the processor core 104 may be a CPU, a GPU, or an accelerator core. The PIM unit 150 is characterized by faster access to data compared to the host processor 132. Thus, the PIM unit 150 may operate at the direction of the processor core 104 to perform memory-intensive tasks. In the example of FIG. 1, the PIM units 150, 152 are coupled to respective memory arrays 182, 192 (e.g., DRAM arrays). The memory arrays 182, 192 may be one or more arrays of memory cells of a bank, channel, or other memory hierarchy partition in the memory device 180. Without loss of generality, in this particular example, it is assumed that each PIM unit 150, 152 is coupled to a respective memory bank 184, 186 that includes the memory arrays 182, 192. Those skilled in the art will appreciate that various configurations of PIM modules and memory partitions (physical or logical) in a PIM-enabled memory device can be used without departing from the spirit of this disclosure. PIM-enabled memory device 180 can be a memory device such as a double data rate (DDRx) memory device, a graphics DDRx (GDDRx) memory device, a low power DDRx (LPDDRx) memory device, a high bandwidth memory (HBM) device, a hybrid memory cube (HMC) device, a non-volatile random access memory (NV-RAM) device, or other memory device that supports PIM.Although the examples herein are provided in the context of a PIM platform, it will be understood that the examples are applicable to other types of memory-centric processing platforms, such as process-near-memory (PNM).

[0018] In some embodiments, the PIM unit 150 includes a control logic 114 (e.g., a command decoder) for decoding instructions or commands issued from the processor core 104, an arithmetic logic unit (ALU) 116 for performing the operations indicated in the PIM instructions, and a PIM register file 118 including multiple indexed registers for holding data for load / store operations to memory or intermediate values ​​of ALU calculations. In some examples, the ALU 116 can perform a limited set of operations for the ALU of the processor core 104, making the implementation of the ALU 116 less complex and more suitable for, for example, in-memory implementation. PIM instructions can move data between the PIM registers and a memory array and can also trigger calculations on this data in the ALU 116. In some examples, the PIM unit 150 also includes a command buffer 122 for storing PIM instructions written into the command buffer 122 by the host device 170. In some examples, the host processor 132 issues PIM instructions to the ALU 116 of each execution unit 150. In embodiments with a command buffer 122, host processor 132 issues a command that includes an index into a line of the command buffer that holds the operation to be performed by ALU 116. In those embodiments with a command buffer 122, the host-memory interface does not require modification with additional command pins to cover all possible opcodes required for PIM operations.

[0019] In some examples, the memory device 180 includes one PIM unit 150, 152 per DRAM component (e.g., bank, channel, chip, rank, module, die, etc.). In some embodiments, PIM instructions issued from the processor core 104 can access data from the DRAM banks by opening / closing rows and reading / writing columns (as traditional DRAM commands do). In these embodiments, each PIM instruction has a target address, which is also used to direct the PIM instruction to the appropriate PIM unit and to the operation to be performed using the data at the target address. The PIM units 150 can operate on separate subsets of the physical address space. For example, when a PIM instruction reaches the execution unit 150, the PIM instruction can be serialized with other PIM instructions and memory accesses to the DRAM that target the same subset of the physical address space.

[0020] For further explanation, FIG. 2 illustrates a block diagram of an example of a PIM-enabled memory system 200 according to an embodiment of the present disclosure. In the example illustrated in FIG. 2, the PIM-enabled memory system 200 is implemented as an HBM system including multiple memory dies 202 (e.g., DRAM cores) stacked on a memory interface die 204 (e.g., a base logic die). The memory die 202 includes multiple memory banks 206 organized into channels or pseudo channels, where the memory banks in a channel or pseudo channel share an I / O bus. In the example illustrated in FIG. 2, the pseudo channel 228 includes several memory banks 206, but the reader will understand that the number of memory banks in a channel or pseudo channel may be selected by the memory system designer. The I / O bus is implemented by TSVs connecting each memory die 202 to the memory interface die 204. The memory interface die 204 is communicatively coupled to a host processor system (e.g., the host device 170 of FIG. 1) through a high-speed link (e.g., an interposer wafer, die-to-die interconnect, etc.). Commands and data are received at memory interface die 204 from a memory controller (e.g., memory controller 140 of FIG. 1) and routed to the appropriate channel or pseudo channel in memory die 202 and to the target memory bank. The commands and data may include PIM commands and host-based data for executing those PIM commands in PIM-enabled memory system 200.

[0021] In some examples, memory bank 206 includes memory array 210, which is a matrix of memory bit cells having word lines (rows) and bit lines (columns), coupled to row buffer 212, which acts as a cache when reading or writing data to / from memory array 210. For example, memory array 210 can be an array of DRAM cells. Memory bank 206 also includes I / O line sense amplifier (IOSA) 214, which amplifies data read from memory array 210 for output to an I / O bus (or to a PIM unit, described below). Memory bank 206 also includes additional components not shown here (for ease of understanding), such as row decoders, column decoders, command decoders, additional sense amplifiers, drivers, signals, and buffers.

[0022] In some embodiments, memory bank 206 includes a PIM unit 226 that performs PIM calculations using data stored in memory array 210. PIM unit 226 includes a PIM ALU 218 that can perform basic calculations within memory bank 206, and a PIM register file 220 that includes a number of PIM registers for storing results of the PIM calculations and for storing data from the memory array and / or host-generated data used as operands for the PIM calculations. PIM unit 226 also includes control logic 216 for loading data from memory array 210 and host-generated data from the I / O bus into PIM register file 220, and for writing result data to memory array 210. Once a PIM calculation or set of PIM calculations is completed, the results in PIM register file 220 are written back to memory array 210. Due to its physical proximity to memory array 210, PIM unit 226 can complete PIM tasks faster than if operand data was sent to the host for calculation and result data was sent back to memory array 210.

[0023] Returning to FIG. 1 , in some embodiments, the processor ISA defines a set of architected PIM registers (e.g., eight indexed registers) that correspond to the PIM register file 118. In some examples, the PIM memory instructions provided by the ISA include a PIM load instruction (e.g., PIM_Ld) for loading data from memory array 182 into a PIM register in PIM register file 118, and a PIM store instruction (PIM_St) for moving data from a PIM register in PIM register file 118 to memory array 182. In some examples, the PIM computation instructions provided by the ISA (e.g., “PIM_Add”, “PIM_Multiply”, etc.) include instructions that execute on PIM ALU 116 using a data store in register file 118. In various examples, the operands of the PIM instructions include architected PIM registers, memory addresses, and values ​​from processor core 104 registers or other core computed values. In some examples, the ISA further provides a PIM read instruction (PIM_Read) that allows memory controller 140 to read data directly from PIM register file 118, and a PIM write instruction (PIM_Write) that allows memory controller 140 to write data directly to PIM register file 118. Typically, such instructions are not used by applications or included in compiled PIM code because they require data to be transmitted over a data bus, thereby obviating the advantages provided by the PIM platform (e.g., minimizing host-memory data traffic, minimizing associated energy costs, and reducing the latency of these operations). Rather, the PIM read and PIM write instructions are typically used for debugging the PIM code.

[0024] In some examples, when the processor core 104 issues a conventional instruction to read or write data to the memory array 182, 192, the instruction is received by the memory controller 140. The memory controller then dispatches a read command or a write command to the memory device 180 in response to the read command or the write command. The memory controller can also dispatch commands to open or close rows in the memory array. In some examples, when the processor core 104 issues a PIM instruction (e.g., a PIM load instruction or a PIM store instruction), the PIM instruction is received by the memory controller 104. The memory controller then dispatches a corresponding PIM command (e.g., a PIM load command or a PIM store command) to a PIM unit (e.g., PIM unit 150 or 152) in the memory device 180 in response to the PIM instruction. In the context of this disclosure, instructions issued by a processor core to read data from or write data to a memory array in memory device 180 are referred to as “non-PIM instructions,” in contrast to PIM instructions issued by processor core 104 that are directed to a PIM unit in memory device 180. As described in more detail below, some embodiments according to the present disclosure provide a memory controller that dispatches PIM commands to memory device 180 in response to receiving a non-PIM instruction from processor core 104.

[0025] As described above, the PIM register file 118 is not used when not performing PIM operations (i.e., the processor is not sending PIM instructions to the PIM unit 150). In accordance with the present disclosure, the memory controller 140 utilizes the PIM register file 118 to expedite non-PIM memory instructions, such as read and write instructions, directed to the memory device 180. As described above, the memory controller 140 dispatches PIM commands in response to receiving a PIM instruction from the processor 132. In accordance with the present disclosure, the memory controller 140 dispatches PIM commands that are not based on a PIM instruction received from the processor 132. That is, the memory controller 140 dispatches PIM commands to manage the PIM register file 118 to expedite the non-PIM instructions. In some examples, the PIM read and PIM write commands, along with the PIM load and PIM store commands, are used by the memory controller 140 to execute non-PIM read and non-PIM write instructions using the PIM register file 118.

[0026] As used herein, "dispatching" a non-PIM instruction or a PIM instruction may be performed by memory controller 140 issuing a memory command over command bus 144 between host device 170 and memory device 180. Thus, dispatching a non-PIM instruction may be interpreted as dispatching a non-PIM command (e.g., a non-PIM read or write command) to a memory device to perform a memory operation specified in a non-PIM instruction received from processor 132. Furthermore, memory controller 140 may dispatch a PIM command to a memory device to perform a particular PIM operation even if a corresponding PIM instruction was not received from processor 132.

[0027] In some embodiments, memory controller logic 130 uses PIM register file 118 to buffer write operations in memory device 180 for non-PIM write instructions. Typically, memory controller 140 drains write instructions from memory controller write queue 110 by switching the memory channel direction to write mode. While memory channel 188 is in write mode, memory controller 140 cannot read data from the memory device. Thus, the amount of time the memory channel spends in write mode is significant as pending read operations are stalled. To reduce the amount of time memory channel 188 spends in write mode, memory controller logic 130 drains write instructions to PIM register file 118 using PIM write commands. For example, if a non-PIM write instruction writes data to a target memory location in memory array 182, memory controller logic 130 instead writes the data to a PIM register using a PIM write command. Memory controller logic 130 then uses a PIM store command to move the data from the PIM register to the target memory location in memory array 182. This may be done opportunistically, such as when a memory channel or memory bank is idle. Additionally, moving non-PIM instruction write data from a PIM register to a target memory location in memory array 182 can be performed while the memory channel is in a read mode, since the data operand is already resident in memory device 180. Writing data for a non-PIM instruction into a PIM register does not require closing a currently open memory row and activating the destination memory row, thereby reducing the amount of time memory channel 188 spends in a write mode.

[0028] The use of PIM registers as write buffers can also reduce read latency, for example through store-to-load transfers. In some examples, memory controller logic 130 reads the buffered results of non-PIM write instructions directly from the PIM registers in response to a non-PIM read instruction. Memory controller logic 130 reads the buffered write data using a PIM read command that does not require closing a currently open memory row. In other examples, memory controller logic 130 expedites non-PIM read instructions by removing the non-PIM write instruction from the critical path of the non-PIM read instruction. For example, if the non-PIM write instruction incurs a row buffer contention with a non-PIM read instruction, memory controller logic 130 drains the non-PIM write instruction to the PIM register file. Consider an example in which two non-PIM write instructions each write data to two DRAM columns A0 and A1 located in DRAM rows 0 and 1, and then a non-PIM read instruction reads data from DRAM column P0 in DRAM row 2. Since all of these accesses are to different DRAM rows, they all incur row buffer contention. The precharge and activation for row 0 and row 1 are on the critical path of the access to column P0. However, the memory controller logic 130 reduces the number of cycles required before the data in column P0 can be read by using a PIM write command to write the data of the non-PIM write instruction to the PIM register file 118. Thus, the non-PIM read instruction that reads data from column P0 in DRAM row 2 finishes sooner because it does not need to open and close the DRAM row for two non-PIM write instructions. Thus, the latency of the more critical read to column P0 is reduced. The write data of the non-PIM write instruction can be drained to the DRAM later, for example, during a bank idle period.

[0029] The use of PIM registers as write buffers also enables write coalescing, which further reduces the amount of time spent in write mode because multiple writes to the same data held in a PIM register do not require closing the currently open memory row and activating the destination row. In some examples, when multiple non-PIM instructions write data to the same memory row, either by overwriting the same column or by writing to different columns of the row, memory controller logic 130 accumulates the write data in the PIM registers until the writes are opportunistically migrated to the memory row in memory array 182 using a PIM store command.

[0030] In some embodiments, memory controller logic 130 uses PIM register file 118 to buffer or stage read operations in memory device 182 for non-PIM read instructions. For example, a non-PIM instruction to read data from one column is often followed by a non-PIM instruction to read data from the next column in a memory row. The PIM registers can be used as an early fetch or pre-fetch buffer for this adjacent data. In some examples, memory controller logic 130 dispatches a non-PIM read command to read data from a column in parallel with a PIM load command to load the next adjacent column in the row into a PIM register. To avoid widening the command bus to accommodate these parallel instructions, in some examples, memory controller logic 130 dispatches a single command over the data bus that combines reading data from one column in the row and loading data from the next column in the row into a PIM register. The data in the PIM register can be read using a PIM read command. If the data held in the PIM register is the last data to be read from the row, the data in the PIM register can be read using a PIM read command, allowing the row to be closed sooner. Thus, the next row can be opened sooner, reducing the read latency of any reads from the next row. Note that the data bus in the memory bank may need to be expanded to accommodate the parallel reading of data from two columns, but the width of the data bus between the host device 170 and the memory device 180 may remain unchanged. To further reduce latency, in some examples, the memory controller logic 130 dispatches a single command that combines a PIM read command to read the data from the PIM register and a precharge command to close the open row. This further reduces the number of cycles before the next row can be activated.

[0031] In some embodiments, memory controller logic 130 reduces read latency by using PIM register file 118 as an early fetch buffer for staging reads. When writing data to the memory array, memory controller logic 130 switches the direction of memory channel 188 to write mode. Thus, during the write phase, memory controller logic 130 does not issue reads to the idle bank because the channel is in write mode and data cannot be read to the memory controller. However, memory controller logic can dispatch a PIM load command over command bus 144, so that no data needs to pass through data bus 146. Thus, in some examples, memory controller logic 130 stages non-PIM read instructions during write mode by dispatching a PIM load command to load data from the idle bank into the PIM registers. Consider an example where a non-PIM read instruction is pending in the memory controller read queue during write mode. Memory controller logic 130 dispatches a PIM load command to load data into the PIM registers from the memory location targeted by the non-PIM instruction. When memory channel 188 is switched back to read mode, memory controller logic 130 uses a PIM read command to read data from the PIM registers instead of reading data from the memory locations identified in the non-PIM instructions. Using a PIM read command does not incur precharge and activate latency on the critical path. This reduces the average memory access latency of non-PIM read instructions, which may improve performance and energy efficiency. Furthermore, this flow may be non-speculative in embodiments where the memory controller logic only dispatches PIM load commands to fetch data into the PIM registers based on read traffic in the memory controller queue.

[0032] In some embodiments, memory controller logic 130 uses PIM register file 118 as a memory-side cache or prefetch buffer. For example, memory controller logic 130 can pre-populate PIM register file 118 with data loaded using PIM load commands based on a speculative algorithm. If a non-PIM read instruction hits in the memory-side cache, memory controller logic can use a PIM read command to read the requested data from PIM register file 118, which is faster than reading from the memory array because it does not require opening a memory row.

[0033] In some embodiments, memory controller 140 includes a PIM register mapping table 142 to facilitate use of PIM register file 118 to expedite non-PIM instructions. PIM register mapping table 142 maps memory locations to PIM registers. For example, to utilize a PIM register as a write buffer for non-PIM instructions, memory controller logic 130 remaps the write data from the target memory location of the non-PIM write instruction to the PIM register. Memory controller logic 130 uses a PIM write command to write the write data to a PIM register and updates PIM register mapping table 142 to include an association between the PIM register and the target memory location. Memory controller logic 130 then opportunistically moves the write data from the PIM register to the target memory location based on the association between the PIM register and the target memory location recorded in the PIM register mapping table. Similarly, to utilize a PIM register, for example, as an early fetch or prefetch buffer, memory controller logic 130 loads data from a memory location into a PIM register and updates PIM register mapping table 142 to include an association between the PIM register and the memory location. When a non-PIM read instruction hits PIM register mapping table 142 (i.e., the target memory location of the non-PIM read instruction matches a memory location in PIM register mapping table 142), the source of the non-PIM read instruction is remapped from the target memory location of the non-PIM read instruction to the PIM register associated with that memory location, and data is read from the PIM register using a PIM read command. In some examples, PIM register mapping table 142 includes multiple entries, each of which maps a PIM register index to a memory location.

[0034] Consider an example where each PIM register in a DRAM bank is 32 bytes and each DRAM column is 64 bytes. Thus, in the case of a PIM register file having 2*N registers per DRAM bank, there are at most N PIM register mapping table entries per DRAM bank. Thus, the memory controller logic 130 has to keep track of the physical addresses of M*N PIM registers, where M is the number of DRAM banks per channel. In the case of a DRAM configuration having 16 DRAM banks per channel and 16 registers per PIM register file per bank, the memory controller logic 130 has to maintain a PIM register mapping table containing 16*8 = 128 entries. In such an example, the PIM register mapping table 142 can be organized as a K-way set associative cache (K < N) having a number of sets equal to M. The index bits can be provided by the DRAM bank select bits (from the physical address of the memory instruction). The remaining physical address bits can function as tags. The PIM register index can be stored in each PIM register mapping table entry. In the case of K = N, the PIM register mapping table need not store the PIM register index. This is because the PIM register index is implied by each PIM register mapping table set (due to the one-to-one mapping between the PIM registers across all banks and the PIM register mapping table entries). In some embodiments, each PIM register mapping table entry has a valid bit that is set when the PIM register holds data that has been written to the PIM register via a PIM write command or loaded into the PIM register via a PIM load command. That is, the valid bit indicates whether the PIM register holds valid data for non-PIM instructions.The valid bit is reset whenever data is flushed from a PIM register via a PIM store command or read from a PIM register via a PIM read command, thus freeing the PIM register for reuse.

[0035] For further explanation, FIG. 3 illustrates a flow diagram and memory device illustrating an example method for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host, according to some embodiments of the present disclosure. The example of FIG. 3 includes a memory controller 310 that may be similar in some or all respects to memory controller 140 of FIG. 1, particularly in that memory controller 310 is configured to dispatch PIM load and PIM store commands for moving data between memory locations and PIM registers, and PIM read and PIM write commands for reading data from and writing data to PIM registers. The example of FIG. 3 also includes a PIM-enabled memory device 312 that may be similar in some or all respects to memory device 180 of FIG. 1, particularly in that memory device 312 includes at least one memory array 350 (e.g., a DRAM bank) coupled to a PIM unit 352 that includes a PIM register file 354. A row 356 in memory array 350 may include a range of physical addresses 1-P. For example, the physical address may indicate a row-column address. Although only one row 356 is shown for illustrative purposes, it will be understood that a memory array may include thousands of rows. The PIM register file 354 includes PIM registers 1-R. In some examples, each PIM register 1-R is represented by a PIM register index.

[0036] The example method of FIG. 3 includes identifying 302 a first write instruction 320 for writing first data 322 to a first memory location 324, where the first write instruction 320 is not a processing-in-memory (PIM) instruction. In some examples, the non-PIM write instruction 320 identifies a memory location 324 (e.g., a physical memory address) in a memory array of the memory device 312 and the data 322 (or a pointer to a data buffer holding the data) to be written to the memory location 324. For example, the write instruction 320 may specify the data 322 and a row-column address (e.g., “physical address 1”) in a DRAM bank to which the data 322 is destined. In some examples, the non-PIM write instruction is queued in a write queue when it is received at the memory controller 310. When the write instruction 320 advances to the head of the write queue, the memory controller 310 identifies the memory location 324 associated with the write instruction 320. In some embodiments, identifying 302 the first write instruction 320 includes determining whether the write instruction 320 should be dispatched as a write to a memory location 324 or a write to a PIM register 326.

[0037] In some examples, memory controller 310 determines whether to write data 322 to memory location 324 or to PIM register 326 based on the state of a row buffer in the memory bank that contains memory location 324. For example, if writing to memory location 324 would incur a row buffer penalty, memory controller 310 may determine that data 322 is written to PIM register 326. A row buffer penalty is imposed if memory location 324 of write instruction 320 is located in a different row than the row currently open in the row buffer. For example, if memory device 312 is busy writing data to row 0 and physical address 1 of write instruction 320 is located in row 1, memory controller must wait for memory device 312 to complete the write to row 0, then close row 0, activate row 1, and write data 322 to physical address 1. This process may require several memory cycles. However, the number of memory cycles required to write data 322 to the PIM register is much less since row 0 does not have to be closed and row 1 opened.

[0038] In some examples, the memory controller 310 determines whether to write data 322 to a memory location 324 or to a PIM register 326 based on the memory locations for other write instructions in the write queue. When the write queue contains multiple write instructions targeting the same row, interspersed with write instructions targeting different rows, the memory controller 310 can determine that the write instructions targeting the same row should be coalesced in the PIM register. Although the memory controller 310 must maintain the same address write ordering when dispatching write instructions, the memory controller 310 can group the write instructions by ejecting the write instructions to the PIM register. Thus, write instructions targeting the same row can be sent to the PIM register, and therefore can be later ejected to the same memory row.

[0039] In some examples, the memory controller 310 determines whether to write the data 322 to the memory location 324 or to the PIM register 326 based on the read instruction in the read queue. For example, if the memory controller 310 determines that a read instruction in the read queue reads the result of a write instruction 320, the memory controller 310 can write the data 322 associated with the write instruction 320 to the PIM register 326, thus allowing the pending read instruction to read the data 322 from the PIM register, avoiding opening the memory row, and thus reducing the latency of the read instruction. As another example, if the memory controller 310 determines that a pending read instruction should be expedited, the data 322 of the write instruction 320 can be written to the PIM register 326 so that the memory channel can be switched to read mode sooner than if the data was written to the memory location 324. This is because writing the PIM register 326 does not require closing the current row and activating the row containing the memory location 324.

[0040] 3 also includes writing 304 the first data 322 to the first PIM register 326. In some embodiments, the memory controller 310 writes the first data 322 to the first PIM register 326 using a PIM write command that identifies a register index of the first PIM register 326 and the data 322 to be written to the first PIM register 326. In the example shown in FIG. 3, the memory controller writes the data 322 to PIM register 1.

[0041] In some embodiments, the memory controller 310 writes 304 the first data 322 to the first PIM register 326 during a bank refresh operation. For example, some memory devices implement a bank-by-bank refresh policy. During the refresh phase, the memory controller 310 activates and precharges rows to a row buffer. In such an example, a refresh operation to refresh a set of rows from a bank lasts on the order of a few nanoseconds. During this time, if the channel is in write mode and a non-PIM instruction 320 targets the bank being refreshed, the memory controller 310 uses a PIM write command to write the data 322 to the PIM register 326. In this way, the latency of the refresh operation overlaps with the latency of the PIM write instruction that transfers the data to the PIM register. The overlap is possible because the bank refresh is an internal memory operation and does not use the memory channel (data or command bus). In addition, the PIM write instruction does not use the row buffer that the refresh operation utilizes.

[0042] The method of FIG. 3 also includes moving 306 the first data 322 from the first PIM register 326 to the first memory location 324. In some embodiments, the memory controller 310 uses a PIM store command to move the first data 322 to the first memory location. For example, the PIM store command specifies a register index of the PIM register 326 that holds the data 322 and the memory location 324 that is the destination of the write instruction 320. In one example, moving 306 the first data 322 to the memory location 324 includes closing an open row, activating the DRAM row that contains the memory location 324 (if the DRAM row is not already open), then writing the data 322 to a row buffer, and finally closing the row. For example, as shown in FIG. 3, the DRAM row 356 (containing the memory location 324) is activated and the data 322 is moved from PIM register 1 into the row 356. When a DRAM row 356 is closed, the data 322 remains in the memory location 324 in the row of the DRAM array. In some embodiments, moving 306 the first data 322 to the memory location 324 is performed opportunistically. In one example, the first data 322 is moved to the memory location 324 while the memory bank is in an idle state. In a further example, the first data 322 is moved to the memory location 324 while the memory channel is in a read mode.

[0043] For further explanation, FIG. 4 illustrates a flow diagram illustrating an example method for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host, according to some embodiments of the present disclosure. The method of FIG. 4 is similar to the method of FIG. 3. However, the method of FIG. 4 differs from the method of FIG. 3 in that writing 304 the first data 322 to the first PIM register 326 includes updating 402 the PIM register mapping table 404 to include an entry 406 that associates the first PIM register 326 with the first memory location 324. In some variations, there is a one-to-one mapping between all PIM registers in the memory channel and PIM register mapping table entries. In these embodiments, the memory controller 310 updates the entry 406 corresponding to the PIM register 326 that holds the data 322 for the non-PIM write instruction 320 with the memory location 324 identified in the non-PIM write instruction 320. In other variations where there is no one-to-one mapping, memory controller 310 allocates entry 406 that includes the PIM register index of PIM register 326 and memory location 324. For example, in the example of FIG. 4, memory controller 310 writes data 322 to PIM register 1 and stores the PIM register index of PIM register 1 and physical address 1 in entry 406 of PIM register mapping table 404. In some examples, memory controller 310 sets a valid bit for entry 406 to indicate that entry 406 contains valid write data that must be moved to the memory array. If memory controller 310 determines that the bank is idle and that a valid entry exists in the PIM register mapping table for that bank's PIM registers, memory controller 310 moves the data in those PIM registers to the memory array using a PIM store command, as described above.

[0044] For further explanation, FIG. 5 illustrates a flow diagram illustrating an example method for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host, according to some embodiments of the present disclosure. The method of FIG. 5 is similar to the method of FIG. 4. However, the method of FIG. 5 includes identifying 502 a second entry 506 in a PIM register mapping table 404 that associates a second PIM register 526 with a second memory location 524, where the first memory location 324 and the second memory location 524 are included in one memory row. In some embodiments, the memory controller 310 determines whether two or more valid entries in the PIM register mapping table 404 are associated with memory locations in the same bank row. In such a case, the memory controller 310 moves data in the corresponding PIM register to the bank row during the same row activation phase. In this manner, a row is activated only once to move data for multiple non-PIM write instructions to the memory array. For example, memory controller 310 may identify that multiple non-PIM write instructions in the write queue access the same bank row, but are interspersed with non-PIM write instructions that access different bank rows. In such an example, memory controller 310 may group the instructions to the same bank row by writing the associated data to a PIM register, and then using PIM register mapping table 404 to track the PIM register that holds the data that may be moved during a single row activation. In another example, memory controller 310 may determine that one or more non-PIM write instructions pending in the write queue are directed to the same memory row as data held in a PIM register. In such an example, memory controller 310 may wait for data associated with the pending non-PIM write instructions to be written to a PIM register, and then move the data in both PIM registers to the memory row during the same row activation.

[0045] In yet another example, two or more non-PIM write instructions targeting the same memory location are coalesced in a PIM register such that a first data in a PIM register associated with a first non-PIM write instruction is overwritten with a second data of a subsequent non-PIM write instruction before being moved to a memory location in the bank. For example, the memory controller 310 may determine that a second non-PIM write instruction enqueued after a first non-PIM write instruction overwrites the first non-PIM write instruction. Instead of writing the data of the first non-PIM write instruction to memory, this data is written to a PIM register to allow subsequent writes to the same memory location to be coalesced in the PIM register. In one example, when a non-PIM write instruction is ready for dispatch, the memory controller 310 cross-references the memory location of the ready non-PIM write instruction with other enqueued non-PIM write instructions to determine that the data of the ready non-PIM write instruction should be written to a PIM register. The data of the coalesced non-PIM write instructions in the PIM register is later opportunistically moved to a memory location. In this manner, by using the PIM register to coalesce non-PIM write instructions into the same memory location, multiple separate row activations for each of these non-PIM write instructions are avoided.

[0046] In the example shown in FIG. 5, the PIM register mapping table includes an entry 506 that was added in response to another non-PIM write instruction to write data 522 to memory location 524 (i.e., “physical address 2”). Now, data 522 is being written to a different PIM register 526 (i.e., “PIM register 2”). Thus, entry 506 associates PIM register 2 with physical address 2. In this example, memory controller 310 determines that physical address 1 and physical address 2 are in the same bank row. Thus, data 322, 522 in PIM register 326, 526 can be moved without closing the current row and opening another row.

[0047] 5 also includes moving the second data 522 in the second PIM register 526 to the second memory location 524 during the row activation phase in which the first data 322 is moved to the first memory location 324. In some embodiments, the memory controller 310 simultaneously moves the data 322, 522 in the PIM registers 326, 526 by opening the row containing the memory locations 324, 524, dispatching a first PIM store command that writes the data 322 to the first memory location 324, dispatching a second PIM store command that writes the data 522 to the second memory location 524, and finally closing the row.

[0048] For further explanation, FIG. 6 illustrates a flow diagram illustrating an example method for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host, according to some embodiments of the present disclosure. The example of FIG. 6 includes a memory controller 610 that may be similar in some or all respects to memory controller 140 of FIG. 1, particularly in that memory controller 610 is configured to dispatch PIM load and PIM store commands for moving data between memory locations and PIM registers, and PIM read and PIM write commands for reading data from and writing data to PIM registers. The example of FIG. 6 also includes a PIM-enabled memory device 612 that may be similar in some or all respects to memory device 180 of FIG. 1, particularly in that memory device 612 includes at least one memory array 650 (e.g., a DRAM bank) coupled to a PIM unit 652 that includes a PIM register file 654. A row 656 in memory array 650 may include a range of physical addresses 1-P. For example, the physical address may indicate a row-column address. Although only one row 656 is shown for illustrative purposes, it will be understood that a memory array may include thousands of rows. PIM register file 654 includes PIM registers 1-R. In some examples, each PIM register 1-R is represented by a PIM register index.

[0049] The example method of FIG. 6 includes identifying 602 a first memory location 624 associated with a first read instruction 620, where the first read instruction 620 is not a processing-in-memory (PIM) instruction. In some examples, the non-PIM read instruction 320 specifies a memory location 624 (e.g., a physical memory address) in the memory array 650 from which the data 322 is to be read. In some examples, the non-PIM read instruction is queued in a read queue when it is received at the memory controller 610. When the read instruction 620 is picked by the memory controller 610, the memory location 624 associated with the read instruction 620 is identified and prepared to read the data 622.

[0050] 6 also includes identifying 604 that a PIM register 626 is associated with the first memory location 624. In some embodiments, the memory controller 610 determines whether data targeted by the non-PIM read instruction 620 is stored in a PIM register. For example, the memory controller 610 can track an association between the PIM register 626 and a memory location 624, where the memory location 624 is a source of the data 622 held in the PIM register 626 or a destination of the data 622 held in the PIM register 626. For example, the data 622 held in the PIM register 626 can be data written to the PIM register 626 as part of buffering a non-PIM write instruction directed to the memory location 624, or the data 622 held in the PIM register can be data loaded from the memory location 624 to the PIM register 626 via a PIM load command.

[0051] In some embodiments, the memory controller 610 tracks the association between the PIM register 626 and the memory location 624 using a PIM register mapping table 614 as shown in FIG. 6. However, it will be appreciated that the memory controller 610 may track the association between the PIM register and the memory location, or the data held in the PIM register, through other techniques. In some examples, the PIM register mapping table 614 includes entries that associate the PIM register with a memory location, as described above. In the example of FIG. 6, the entry 616 associates the PIM register 626 with the memory location 624. For example, the entry 616 associates the PIM register index of the PIM register 626 with the physical address or partial physical address (e.g., DRAM row-column address) of the memory location 624. Thus, in some examples, when the memory controller 610 receives a non-PIM read instruction for dispatch to the memory device 612, the memory controller 610 first determines whether the memory location 624 hits the PIM register mapping table 614. Upon identifying the memory location 624 indicated in entry 616, the memory controller 610 determines that the PIM register 626 indicated in entry 616 holds data 622 targeted by the non-PIM read instruction.

[0052] 6 also includes reading 606 the first data 622 from the PIM register 626 in response to the first read instruction 620. In some embodiments, to service the non-PIM read instruction 620, the memory controller 610 reads the data 622 held in the PIM register 626 instead of dispatching a non-PIM read instruction to the memory device 612. In some examples, the memory controller 610 reads the data 622 by dispatching a PIM read command.

[0053] For further explanation, FIG. 7 illustrates a flow diagram illustrating an example method for leveraging processing-in-memory (PIM) resources to expedite non-PIM instructions executed on a host, according to some embodiments of the present disclosure. The method of FIG. 7 is similar to the method of FIG. 6. However, the method of FIG. 7 includes loading 702 a first data 622 from a first memory location 624 into a PIM register 626. Before reading data from the PIM register to service a non-PIM read instruction, the data is first loaded into the PIM register. In some embodiments, the memory controller 610 dispatches a PIM load command to the memory device 612. The data 622 can be loaded into the PIM register 626 under various circumstances to expedite a non-PIM read instruction 620.

[0054] In some examples, the memory controller 610 identifies a non-PIM read instruction 620 as a pending instruction in the memory controller read queue while the memory channel is in write mode. To reduce the read latency of the non-PIM read instruction 620, in these examples, the memory controller 610 loads 702 data 622 from a first memory location 624 into a PIM register 626 while the memory channel is in write mode (i.e., the memory controller 610 is actively dispatching write commands and write data in the memory channel). Because a PIM load command does not require the transfer of read data over the data bus, the memory controller can dispatch the PIM load command while the memory channel is in write mode. In some examples, the data 622 is loaded into the PIM register 626 when the DRAM bank containing the memory location is idle (i.e., the memory controller 610 is not writing data to the DRAM bank). When the memory channel changes to a read mode, the data 622 is read 606 from the PIM register 626 without incurring the latency associated with closing an open row and activating the row that contains the memory location. Thus, the PIM register 626 acts as a staging buffer for non-PIM read instructions, thereby improving performance and energy efficiency.

[0055] In some examples, the memory controller 610 loads 702 data 622 from memory location 624 into PIM register 626 based on a guess instead of the presence of a non-PIM read instruction 620 in the memory controller read queue. For example, the memory controller 610 can use various heuristics to guess which memory location should be the target of the PIM load command. In one example, the memory controller 610 can track the pattern of columns accessed by read instructions for the same memory row and issue a PIM load command to prefetch one or more of the remaining columns of the memory row into the PIM register. If the column address pattern of the read instructions is accessing the same row in consecutive column order, one heuristic is to prefetch the next consecutive unread column of the row into the PIM register before closing the open row. In another example, the memory controller 610 preloads the PIM register with data from memory banks of frequently accessed memory rows. For example, the memory controller 610 can use a table to track frequently accessed rows per bank. Before the memory controller 610 closes a frequently accessed memory row, the memory controller can dispatch a PIM load command to copy unread columns from the row into the PIM registers. The memory controller 610 can use the next consecutive column heuristic described above or other prefetch algorithms recognized by those skilled in the art.

[0056] The method of FIG. 7 also includes updating 704 the PIM register mapping table 614 to include an entry 616 that associates the PIM register 626 with the first memory location 624. In some embodiments, upon loading the data 622 into the PIM register 626, the memory controller 610 indicates in the PIM register mapping table 614 that the PIM register 626 holds the data 622 associated with the memory location 624, as described above. For example, the entry 616 may include a PIM register index for the PIM register 626 and a physical address or partial physical address (e.g., memory bank row-com address) of the memory location 624. In the example of FIG. 7, the entry 616 indicates that the PIM register 1 holds data associated with physical address 2 (i.e., memory location 624). In some examples, the memory controller 610 marks the entry 616 as holding valid read data.

[0057] For further explanation, FIG. 8 illustrates a flow diagram illustrating an example method for leveraging processing-in-memory (PIM) resources to accelerate non-PIM instructions executed on a host, according to some embodiments of the present disclosure. The method of FIG. 8 is similar to the method of FIG. 7. However, the method of FIG. 8 also includes reading second data 822 from a second memory location 824, where the first memory location 624 and the second memory location 824 correspond to consecutive columns in a memory row. In some examples, the memory controller 610 loads 702 the data 622 from the first memory location 624 in the first column into the first PIM register 626 based on a preceding non-PIM read instruction that reads data from the second memory location 824 in an adjacent column of the memory row 656. Consider an example where there are two non-PIM read instructions pending in the read queue that access consecutive columns A1 and A2 (e.g., physical address 1 and physical address 2) of memory row 656, and then another read instruction to a different memory row that requires closing row 656. Typically, a memory controller would dispatch a command to open row 656, then dispatch a read command to column A1, then dispatch a read command to column A2, then close row R. Now, memory controller 610 dispatches a command to open row 656, then dispatches a command that combines a read of data 822 column A1 and a PIM load command to load data 622 from column A2 into PIM register 626, then close row 656. Thus, row 656 can be closed sooner than in the typical case, allowing a new row to be activated sooner than in the typical case, and thus reducing the overall latency of the read instruction accessing the new row. Here, we assume that the data bus in the memory bank supports parallel transfer of data from two columns, one over the host memory data bus and the other to the PIM registers, however, only one column is transferred over the host-memory data bus, thus not requiring an increase in bus lanes.

[0058] In one variation of the above, to avoid widening the command bus to send non-PIM read commands (i.e., commands to read from the bank array) and PIM load commands in parallel, the non-PIM read command and the PIM load command can be combined into a single special command. Thus, when the special read command reads data from a column in a memory bank row, the data in the next consecutive column is loaded into the specified PIM register. This does not require additional column address bits because data is always fetched from the next consecutive column. Thus, in some examples, this instruction is used to read data 822 from memory location 624 over the data bus while simultaneously loading data 622 from memory location 824 into PIM register 626.

[0059] In another variation of the above, the PIM write command to read data 622 from the PIM register is combined with a command to close the memory row in order to close the memory row early. For example, the memory controller 610 can issue a special command to close the row 656 while simultaneously reading data 622 from the PIM register 626. This operation is possible because the data 622 has already been read from the row 656 into the PIM register. Closing the open row in parallel with reading data from the PIM register allows the new row to be activated earlier than in the typical case described above.

[0060] In view of the above disclosure, the reader will appreciate that embodiments of the present disclosure support using idle PIM resources to expedite non-PIM instructions. The use of PIM registers as write buffers increases the rate at which write instructions can be drained from the memory controller write queue, thereby reducing the amount of time the memory channel spends in write mode. This in turn expedites switching the memory channel to read mode to process pending non-PIM read instructions. The use of PIM registers as read buffers reduces read latency by transferring data to the PIM registers before dispatching the non-PIM read instructions, because the non-PIM read instructions do not need to open the row when they are dispatched by the memory controller. Additionally, the grouping of data transfers between the PIM registers and the DRAM when a memory row is activated saves energy and improves performance.

[0061] Some embodiments may be systems, devices, methods, and / or logic. The computer-readable program instructions of the present disclosure may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages, and functional programming languages. In some embodiments, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute the computer-readable program instructions by using state information of the computer-readable program instructions.

[0062] Aspects of the present disclosure are described herein with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and logic circuitry according to some embodiments of the present disclosure. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by logic circuitry.

[0063] The logic circuitry may be implemented in a processor (e.g., a CPU, GPU, or accelerator), other programmable data processing apparatus, or other device to cause a series of operational steps to be executed on the processor, other programmable apparatus, or other device to generate a computer-implemented process such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / operations specified in one or more blocks of the flow diagrams and / or block diagrams.

[0064] The flow diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and logic circuits according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flow diagrams, as well as combinations of blocks in the block diagrams and / or flow diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or acts, or a combination of dedicated hardware and computer instructions.

[0065] Although the present disclosure has been specifically shown and described with reference to embodiments thereof, it should be understood that various changes in form and details may be made therein without departing from the spirit and scope of the following claims. Accordingly, the embodiments described herein are merely illustrative and not limiting of the present invention. The present disclosure is defined by the appended claims, not the detailed description, and all differences within the scope thereof should be construed as being included in the present invention.

Claims

1. 1. A method comprising: writing first data of a first write instruction to a first processing-in-memory (PIM) register, the first write instruction being not a PIM instruction; moving the first data from the first PIM register to a first memory location; method.

2. identifying the first write instruction for writing the first data to the first memory location; 2. The method of claim 1.

3. the first data is moved to the first memory location during a bank idle period; 2. The method of claim 1.

4. the first data is moved to the first memory location while the memory channel is in a read mode; 2. The method of claim 1.

5. the first data is written to the first PIM register by dispatching a PIM write command, and the first data is stored to the first memory location by dispatching a PIM store command.

2. The method of claim 1.

6. Writing the first data to the first PIM register includes: updating a PIM register mapping table to include a first entry that associates the first PIM register with the first memory location; 2. The method of claim 1.

7. identifying a second entry in the PIM register mapping table that associates a second PIM register with a second memory location, the first memory location and the second memory location being included in one memory row; During a row activation phase in which the first data is moved to the first memory location, moving second data in the second PIM register to the second memory location. The method of claim 6.

8. 1. A method comprising: receiving a first read instruction associated with a first memory location, the first read instruction being not a processing-in-memory (PIM) instruction; reading first data from a PIM register in response to the first read command; method.

9. identifying the PIM register as associated with the first memory location; 9. The method of claim 8.

10. an entry in a PIM register mapping table associates the first PIM register with the first memory location; 9. The method of claim 8.

11. The first data is read from the PIM register by dispatching a PIM read command.

9. The method of claim 8.

12. loading the first data from the first memory location into the PIM register; updating a PIM register mapping table to include an entry associating the PIM register with the first memory location.

9. The method of claim 8.

13. the first data is loaded into the PIM register by dispatching a PIM load command; 13. The method of claim 12.

14. the first data is loaded into the PIM register while the memory channel is in a write mode, and the first data is read from the PIM register while the memory channel is in a read mode.

13. The method of claim 12.

15. the first data is speculatively loaded into the PIM register; 13. The method of claim 12.

16. reading second data from a second memory location; the first memory location and the second memory location correspond to successive columns of a memory row; the first data is loaded into the PIM register from the first memory location while the second data is read from the second memory location; 13. The method of claim 12.

17. a single command is dispatched to read the second data from the second memory location and to load the first data from the first memory location; 17. The method of claim 16.

18. a single command is dispatched to read the first data from the PIM register and close the memory row; 17. The method of claim 16.

19. An apparatus comprising: A processor; a memory controller including logic; The logic comprises: writing first data of a first write instruction to a first processing-in-memory (PIM) register, the first write instruction being not a PIM instruction; moving the first data from the first PIM register to a first memory location; [0023] Device.

20. the memory controller including logic configured to identify the first write instruction for writing the first data to the first memory location; 20. The apparatus of claim 19.

21. Writing the first data to the first PIM register includes: updating a PIM register mapping table to include a first entry that associates the first PIM register with the first memory location; 20. The apparatus of claim 19.

22. the first data is written to the first PIM register during a bank refresh period; 20. The apparatus of claim 19.

23. the first data is moved to the first memory location while the memory channel is in a read mode; 20. The apparatus of claim 19.

24. An apparatus comprising: A processor; a memory controller including logic; The logic comprises: receiving a first read instruction associated with a first memory location, the first read instruction being not a processing-in-memory (PIM) instruction; reading first data from a PIM register in response to the first read command; [0023] Device.

25. an entry in a PIM register mapping table associates the first PIM register with the first memory location; 25. The apparatus of claim 24.

26. the memory controller includes logic; The logic comprises: loading the first data from the first memory location into the PIM register; updating a PIM register mapping table to include an entry associating the PIM register with the first memory location; [0023] 25. The apparatus of claim 24.

27. the first data is loaded into the PIM register while the memory channel is in a write mode, and the first data is read from the PIM register while the memory channel is in a read mode.

27. The apparatus of claim 26.

28. 1. A system comprising: A processor; a memory controller including logic; A processing-in-memory (PIM) compatible memory, The logic comprises: reading first data from a PIM register of the PIM enabled memory in response to a read instruction associated with a first memory location, the read instruction being not a PIM instruction; in response to a write instruction associated with a second memory location, writing second data of the write instruction to a second PIM register, the write instruction being not a PIM instruction; moving the second data from the second PIM register to the second memory location; [0023] system.

29. the first data is loaded into the PIM register while the memory channel is in a write mode, and the first data is read from the PIM register while the memory channel is in a read mode.

30. The system of claim 28.

30. the second data is written to the second PIM register during a bank refresh period; 30. The system of claim 28.

31. the second data is moved to the second memory location while the memory channel is in a read mode.

30. The system of claim 28.

Images