Memory access commands with near memory address generation

Abstract

The memory controller may be configured with command logic capable of sending memory access commands with incomplete address information over a command / address bus connecting the memory controller to the memory modules. The memory controller may send the memory access commands over the bus to access data stored in memory locations of the memory modules. The memory locations may correspond to different near memory addresses generated to reflect that the data is not address-aligned across the memory modules. Nevertheless, due to the near memory address generation, the memory controller can send memory access commands with incomplete address information to access data stored at different addresses, as opposed to having to send multiple memory access commands over the bus specifying complete address information to access data at different addresses, thereby conserving use of available bus bandwidth, reducing power consumption, and increasing computational throughput.

Description

Technical Field

[0001] Disclosed embodiments of memory access commands having near-memory address generation generally relate to volatile and non-volatile computer memory, and more particularly, to computer-implemented logic for accessing data stored in computer memory.

Background Art

[0002] Computer memory is a fundamental component of substantially all computers, including personal computers, tablet computers, smartphones, server computers, and other computing devices such as, for example, printers and Internet-connected devices. Computer memory is typically used in a computer to store data and processor-executable instructions (machine code) that operate on the data.

[0003] The term "memory" is often used to refer to volatile computer memory such as, for example, dynamic random access memory (or simply "DRAM" for brevity) in the form of an integrated circuit chip having metal-oxide-semiconductor memory cells, but computer memory may be non-volatile. A non-exhaustive list of volatile and non-volatile computer technologies includes DRAM, static random access memory (or simply "SRAM" for brevity), non-volatile dual in-line memory module (or simply "NVDIMM" for brevity), flash memory, embedded DRAM, scratchpad memory, and the like.

[0004] A computer's computer memory is typically connected to the computer's hardware "host" via a set of computer wires or conductors. The hardware host can include a memory controller and a central processing unit (or simply "CPU" for brevity), a graphics processing unit (or simply "GPU" for brevity), or other hardware data processors. The connection between the computer memory and the host typically includes an interface for command / address information (sometimes called the "command / address bus") and an interface for data information (sometimes called the "data bus").

[0005] In recent years, the computing power and speed of processors (e.g., CPUs and GPUs) have been increasing faster than the bandwidth of the data bus connecting the computer memory to the host. As a result, the data bus has become a bottleneck in computing throughput because the processor wastes time waiting for data to be transferred between the computer memory via the data bus.

[0006] Processor-in-memory (or simply "PIM") solutions provide data processing capabilities close to the computer memory (e.g., on the same die). In PIM solutions, since the data processing capabilities are close to the computer memory, calculations can be performed on the data accessed from the computer memory without the need to transfer all of the raw data to the host via the data bus. For example, a possible PIM solution can include adding simple vector calculation elements to each internal memory module of the computer memory, such as each DRAM bank or sub-array. These additional elements then allow the host to trigger calculations in all internal memory modules without the need to transfer data across the data bus and across the DRAM interface.

[0007] The PIM solution can save the consumption of the data bus bandwidth due to data transfer between the computer memory and the host, and among other types of compute-intensive workloads that may reach the data bus bandwidth limit by data transfer between the computer memory and the host in the absence of the memory-local PIM unit, particularly for the increasing category of data-bound compute workloads such as, for example, deep artificial neural network machine learning workloads, large-scale graph analysis workloads, sparse computations in machine learning workloads, etc., the throughput of the compute workload can be improved.

[0008] As a mere example, a common operation in machine learning is sparse embedding lookup. This operation can dominate the execution time of some workloads. Such an operation can sparsely read out large embeddings containing hundreds of elements from a large embedding table containing millions of embedding entries stored in the computer memory and accumulate them together. Since data reuse is limited, such an operation can be very sensitive to the data bus bandwidth.

[0009] When not using the PIM solution, the sparse embedding lookup operation may require the host to issue a set of memory load requests, one for each of several memory modules (e.g., DRAM banks). With each memory load request, data is transferred from each memory module across the data bus to the host. Since the memory modules can share the command / address bus and the data bus (e.g., DRAM banks on the same memory channel), accesses to different memory modules may need to be serialized to result in full utilization of both the command / address and data buses. In contrast, when the PIM solution is used, the accumulation can be computed in the memory-local PIM unit without the need to transfer some of the data on the data bus.

[0010] However, since the PIM solution can alleviate some or all of the bottleneck in the data bus of a particular workload, the bottleneck in the command / address bus can become a new bottleneck. For example, the bottleneck in the command / address bus can occur in data-limited compute workloads that have sparse or irregular memory access patterns such as those seen in machine learning workloads and other workloads. Since the command / address bus is shared by memory modules, each memory load command still needs to be transmitted across the command / address bus.

[0011] Possible solutions to the bottleneck in the command / address bus are to increase the bandwidth of the command / address bus by increasing the number of memory channels or by increasing the number of command / address wires per memory channel. However, this increases the manufacturing cost per unit and increases the power consumption. Further, in workloads with normal memory access patterns, the extra command / address bus bandwidth may not be utilized.

[0012] The disclosed embodiments address this and other problems.

[0013] The approaches described in this section are approaches that can be implemented, but are not necessarily approaches that have been previously conceived or implemented. Thus, unless otherwise indicated, none of the approaches described in this section should be assumed to be eligible as prior art or to be well-understood routine or conventional merely by virtue of their inclusion in this section.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0015] Not all of the components shown in each figure are required, and the embodiments may include additional components not shown in the figures. Without departing from the scope of the present disclosure, the arrangement and type of components can be changed. Within the scope of the present disclosure, additional components, different components, or fewer components can be utilized.

[0016] In the following description, for purposes of explanation, in order to provide a thorough understanding of embodiments of memory access commands involving near memory address generation, many specific details are set forth. It will be apparent, however, that embodiments may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the embodiments.

[0017] (Overview) A system and method for memory access commands involving near memory address generation are disclosed. Such a system and method can facilitate computer memory access (e.g., efficiently) to support computational workloads including, but not limited to, deep learning workloads (such as using deep neural networks), large-scale graph analysis, and / or other computational workloads having sparse or irregular memory access. The techniques of the invention can be utilized in a variety of systems including, but not limited to, Northbridge chipset, field programmable array (or simply "FPGA" for brevity) devices, microprocessor devices, CPU devices, GPU devices, memory integrated circuits, and / or other systems such as machine learning and / or artificial intelligence accelerator devices.

[0018] A memory controller configured with command logic can transmit a memory access command having incomplete address information via a connection between the memory controller and two or more computer memory modules. The incomplete address information may not have any address information (addressless). Instead, the incomplete address information may have partial address information. The memory controller transmits the memory access command via a connection for accessing data stored at two or more memory locations of the memory module. The memory locations correspond to two or more near memory generated addresses.

[0019] The near-memory generation address corresponding to the memory location may vary, reflecting that the data is not address-aligned across the memory modules. Nevertheless, for near-memory address generation, in contrast to the need for the memory controller to send multiple memory access commands on the bus specifying complete address information for accessing data at different addresses, it can send a memory access command with incomplete address information for accessing data stored at different addresses, thereby saving the use of available bus bandwidth, reducing power consumption, and increasing computational throughput.

[0020] The command logic of the memory controller is configured to send a program address command specifying an address to the memory module via the connection. These pre-programmed addresses can be stored in the near-memory, can be different for each different memory module, and can be utilized across multiple memory access commands with incomplete address information, thereby avoiding the need to send complete address information via the connection with each memory access command.

[0021] A memory access command with incomplete address information can specify an operation (e.g., multiplication or addition) to be performed by the PIM unit of the memory module. For near-memory address generation, the transmission of a single memory access command with incomplete address information enables the PIM unit to perform operations on data stored in different memory modules even when the data is not address-aligned across the memory modules.

[0022] Any one of the memory modules has a memory structure and a near-memory address generation unit. The memory structure includes memory locations that can be addressed. The near-memory address generation unit is configured to generate an address corresponding to the memory location of the memory structure in the near-memory. The near-memory address generation unit is configured to generate an address based on the reception in the memory module via a connection of a memory access command having incomplete address information transmitted by a memory controller via a connection to access data stored at the memory location of the memory module corresponding to the near-memory generated address.

[0023] The near-memory address generation unit includes a register file. The register file is configured to store the address generated by the near-memory address generation unit. The address is stored in the register file based on the reception in the memory module via a connection of a program address command specifying the address stored in the register file.

[0024] The near-memory address generation unit includes an address calculation unit configured to calculate the address generated by the near-memory address generation unit. The address calculation unit calculates the generated address based on applying a pre-configured stride to the address. The address calculation unit calculates the generated address based on a value obtained from the PIM unit of the memory module or based on a value obtained from the memory location of the memory structure.

[0025] The memory controller comprises command logic configured to transmit a first memory access command having incomplete address information via a connection for accessing data stored in a set of two or more rows of a set of two or more memory modules corresponding to a set of two or more near-memory generation row addresses. Further, the command logic is configured to transmit a second memory access command having incomplete address information via a connection for accessing data stored in a set of two or more columns of a set of two or more memory modules corresponding to a set of two or more near-memory generation column addresses.

[0026] The memory module includes a memory structure having a plurality of rows and a plurality of columns. Further, the memory module includes a near-memory address generation unit configured to generate a row address corresponding to any one of the plurality of rows based on reception by the module via a connection of a first memory access command for accessing data stored in the row. The near-memory address generation unit is further configured to generate a column address corresponding to any one of the plurality of columns based on reception by the module via a connection of a second memory access command for accessing data stored in the column of the row.

[0027] Embodiments support (e.g., efficiently support) a CPU, a GPU, and a PIM unit when processing a computational workload involving irregular or sparse data access to computer memory. Such workloads include deep learning inference models and large-scale graph analysis that are becoming increasingly prevalent in the commercial, government, education, and research fields. Embodiments enable such workloads to access memory faster and with lower power consumption.

[0028] These and other embodiments are described in more detail with reference to the drawings.

[0029] (System for Memory Access Commands with Near-Memory Address Generation) FIG. 1 is a schematic diagram of an exemplary system for a memory access command involving near memory address generation. The system includes a memory controller 102 electrically connected to a command / address bus 106 shared by two or more memory modules 108, and the memory modules 108 are also electrically connected. The electrical connection may be direct or indirect, and an indirect electrical connection may include additional structures within the electrical path.

[0030] The system can include other components not shown in FIG. 1. For example, the system can include other buses, chips, and circuits not shown in FIG. 1.

[0031] In the example of FIG. 1, three memory modules are shown. However, the system can have as few as two memory modules, or three or more memory modules including four or more memory modules. In addition to the command logic 104, the memory controller 102 can include other logic and structures not shown in FIG. 1, such as a request buffer, a response buffer, memory mapping logic, and arbitration logic. In addition to being electrically connected to the memory modules 108 by the command / address bus 106, the memory controller 102 may also be electrically connected to the memory modules 108 by other buses not shown in FIG. 1, such as a data bus.

[0032] Memory controller 102 can include hardwired and / or programmed digital circuitry for managing the flow of data to and from memory module 108. Memory controller 102 can be an individual chip or can be integrated into another chip, such as on the same die or as an integrated component of a microprocessor such as a CPU or GPU. In addition to command logic 104, memory controller 102 can include logic for reading data from and writing data to memory module 108. If memory module 108 is a volatile memory device, memory controller 102 can include logic for periodically refreshing the charge in the memory cells of memory module 108, such as when memory module 108 is implemented based on semiconductor DRAM.

[0033] By sharing command / address bus 106, commands sent by memory controller 102 for one, two or more, or all of memory modules 108 can be sent via command / address bus 106. When a command is sent from memory controller 102 to one or more of memory modules 108 via command / address bus 106, the command can be sent from memory controller 102 to one or more memory modules along only command / address bus 106 or along one or more additional electrical connections within the electrical path between memory controller 102 and one or more memory modules.

[0034] Command / address bus 106 can include a set of one or more wires or conductors for transmitting electrical signals representing commands including program address commands and memory access commands having incomplete address information disclosed herein. The electrical signals can be transmitted from memory controller 102 to memory module 108 via command / address bus 106.

[0035] The connection between the memory 102 controller and the memory module 108 can include an interface for transmitting and receiving command / address information such as, for example, a memory access command. This connection is labeled as the command / address bus 106 in FIG. 1. The same connection or another connection between the memory controller 102 and the memory module 108 can include an interface for transmitting and receiving data such as, for example, data accessed from a memory location. This connection may be referred to herein as the "data bus". Although the term "bus" is used, one or more connections can include a set of wires (s) or conductors (s) between the memory controller 102 and the memory module 108, or other forms of wired and / or wireless electrical couplings or connections (s). Further, as illustrated, the command / address bus 106 and the data bus may share the same electrical connection between the memory controller 102 and the memory module 108, or individual electrical connections may be used.

[0036] The command / address bus 106 may be composed of sub-buses. For example, the command / address bus 106 may be composed of a row address bus and a column address bus, potentially among other sub-buses.

[0037] The memory module 108 provides addressable memory locations for storing data. In some cases, the data stored in or accessed from a memory location is referred to herein as a "value" or "data value". Such values may be represented by one or more bits. In some cases, the memory location is byte-addressable and stores an 8-bit data value. In some cases, the memory location is word-addressable and stores a multi-byte data value (e.g., a 16-bit, 32-bit, 64-bit, or 128-bit data value).

[0038] When a data value is accessed from a memory location (e.g., 110) of a memory module (e.g., 108-0) by a command sent by the memory controller 102, the data value can be sent back from the memory module to the host via a data bus between the memory module 108 and the host. Additionally, or alternatively, the data value may be sent to a PIM unit near the memory module for calculation or accumulation (e.g., multiplication, addition, etc.) by the PIM unit.

[0039] As used herein, the term "near memory" includes memory within the same die, chip, or circuit as a memory module (e.g., 108-0). However, more broadly, near memory includes structures that are physically closer to the memory module than the host or the memory controller 102 at the other end of the command bus 106 for the memory module.

[0040] Each memory module 108-0, 108-1, 108-2 is a DRAM bank. However, each memory module 108 may correspond to another type of component within a memory hierarchy such as a subarray, rank, or channel. Further, the memory module is not limited to a volatile memory device or a DRAM device and can include a non-volatile memory device or other type of memory device having addressable memory locations.

[0041] The near memory address generation unit is used to generate near memory addresses. For example, individual near memory address generation units can be placed within each of the memory modules 108-0, 108-1, and 108-2. For example, if the memory module 108 is a component of a stack of multiple memory layers on a base die, the near memory address generation unit can be placed within the base die, or the near memory address generation unit can be placed within a memory layer of a three-dimensional stacked memory or within a memory layer of a more conventional DRAM.

[0042] The near-memory address generation unit can also be arranged in memories in other forms than 3D stacked DRAM and conventional DRAM (for example, non-volatile memory, flash memory, embedded DRAM). Although the embodiments described below are related to the DRAM memory structure, the near-memory address generation unit and its functions can be implemented in other memory contexts such as other emerging memory technologies or later-developed memory technologies including non-volatile memory modules such as NVDIMM and other forms of memory such as scratchpad memory.

[0043] (Address-aligned data) The near-memory address generation unit associated with the memory module 108 enables the memory controller 102 to send a memory access command having incomplete address information such as no address information or only partial address information. Nevertheless, the near-memory address generation unit can generate an address near the memory to provide complete address information, either completely or in combination with the partial address information specified in the memory access command. By doing so, computational workloads such as machine learning and large-scale graph analysis and other computational workloads that result in irregular memory accesses reduce the consumption of the bandwidth of the command / address bus 106 and are supported with lower power consumption.

[0044] For example, for a near-memory address generation unit associated with memory module 108, a memory access command having incomplete address information can be sent from memory controller 102 to two or more memory modules via command / address bus 106 to access a memory location within a different memory module 108 of memory module 108 even if the memory location is not address-aligned. For example, memory location 110 of memory module 108-0 is designated as having the address "0:2" in FIG. 1. This notation is used for illustrative purposes only. The number before the colon ":" designates (e.g., identifies) the memory module of memory module 108 connected to command 106 (in this example, "0"). The number after the colon ":" designates the memory location within the addressed memory module (in this example, "2"). In some cases, the portion of the address that designates a particular memory location within the addressed memory module is referred to herein as the "module-relative" address. In this example, the "2" following the colon is the module-relative address of the address "0:2".

[0045] Here again, it should be emphasized that the address notation in FIG. 1 is used for illustrative purposes only in this disclosure. A practical address may be a multi-bit address having a plurality of components such as some bits that designate a memory module, some bits that designate a row address of the addressed memory module, and some bits that designate a column address of the addressed memory module, among other bit representation information. Other multi-bit address representations are possible and no particular address representation is required.

[0046] Some computing workloads exhibit normal memory access patterns where memory accesses are address-aligned across memory module 108. For example, the memory location 110 at address "0:2" in memory module 108-0 is address-aligned with the memory location at address "1:2" in memory module 108-1, which is in turn address-aligned with the memory location at address "2:2" in memory module 108-2, because all three memory locations have the same module-relative address (in this example, "2").

[0047] When data accessed from two or more memory modules is address-aligned, a single memory access command with complete address information can be sent to access multiple address-aligned memory locations. For example, the memory controller 102 can send a single memory access command that specifies "2" as the module-relative address (e.g., broadcast) to access the data stored at the memory locations "0:2", "1:2", and "2:2". For example, a single memory access command can be broadcast by the memory controller 102 on the command / address bus 106 to all memory modules 108 that specify "2" as the module-relative address that uniquely identifies the memory location to be accessed in each of the memory modules 108.

[0048] However, when data accessed in two or more of the memory modules 108 is not address-aligned, multiple memory access commands may need to be sent by the memory controller 102. For example, if the data being accessed is stored at the memory locations "0:2" and "1:3", two memory access commands specifying different addresses may need to be sent by the memory controller 102 via the command / address bus 106.

[0049] However, using the near-memory address generation unit associated with the memory module 108, a single memory access command having incomplete address information can be transmitted by the memory controller 102 via the command / address bus 106 to access data stored at different misaligned memory locations across two or more of the memory modules 108. For example, the command can access data stored at memory locations "0:2", "1:3", and "2:0", and some or all of the module-relative address can be generated by the near-memory address generation unit associated with the memory module 108. For example, a first near-memory address generation unit associated with the memory module 108-0 can generate a module-relative address "2" based on the memory module 108-0 receiving a command from the memory controller 102 via the command bus 106. A second near-memory address generation unit associated with the memory module 108-1 can generate a module-relative address "3" based on the memory module 108-1 receiving a command from the memory controller 102 via the command bus 106. A third near-memory address generation unit associated with the memory module 108-2 can generate a module-relative address "0" based on the memory module 108-2 receiving a command from the memory controller 102 via the command bus 106.

[0050] (Incomplete address information) As used herein, a memory access command having "incomplete" address information includes a command transmitted by the memory controller 102 to two or more memory modules 108 via the command / address bus 106, and part or all of the module-relative address is generated in the near memory. As described above, the module-relative address can include a portion of the complete address that identifies the memory location of the memory module addressed by the complete address accessed by the command. In contrast, a memory access command having "complete" address information can include a command transmitted by the memory controller 102 to two or more memory modules 108 via the command / address bus 106, the entire module-relative address is specified by the command, and no portion of the module-relative address part is generated in the near memory.

[0051] A memory access command having incomplete address information may not specify any portion of the module-relative address. In this case, the entire module-relative address can be generated in the near memory. For example, a memory access command having incomplete address information may not specify a row address and may not specify a column address. In this case, the row and column addresses may be generated in the near memory by the near memory address generation unit. Note that for the near memory address generation unit, different row and column addresses can be generated in the near memory for different memory modules being accessed, such as when the data is not address-aligned across the memory modules. And when the data is not address-aligned across the memory modules being accessed, for the near memory address generation unit, the memory controller 102 only needs to transmit a single memory access command via the command / address bus 106, and does not need to transmit a separate memory access command for each different module-relative address, thereby saving the bandwidth and power consumption of the command / address bus 106.

[0052] A memory access command having incomplete address information may specify only partial address information rather than all of the module-relative address. For example, a memory access command having incomplete address information may specify only the row address or only the column address common to the memory module being accessed, without specifying both the row address and the column address, or may specify a complete address or all of the module-relative address. In this case, the missing portion of the module-relative address may be generated in the near memory.

[0053] For example, a memory access command having incomplete address information may specify a row address command for the memory module being accessed but may not specify a column address. In this case, the column address may be generated in the near memory by the near memory address generation unit. Note that for the near memory address generation unit, different column addresses can be generated in the near memory for different memory modules being accessed, such as when data is not address-aligned across the memory modules. And when data is not address-aligned across the memory modules being accessed, for the near memory address generation unit, the memory controller 102 only needs to send a single memory access command via the command / address bus 106, and does not need to send a different memory access command for each different column address, thereby saving the bandwidth and power consumption of the command / address bus 106.

[0054] As another example, a memory access command having incomplete address information may specify a column address command for the memory module being accessed but may not specify a row address. In this case, the row address may be generated in the near memory by the near memory address generation unit. Note that for the near memory address generation unit, different row addresses can be generated in the near memory for different memory modules being accessed, such as when data is not address-aligned across the memory modules. And when data is not address-aligned across the memory modules being accessed, for the near memory address generation unit, the memory controller 102 only needs to send a single memory access command via the command / address bus 106, and there is no need to send a different memory access command for each different row address, thereby saving the bandwidth and power consumption of the command / address bus 106.

[0055] A memory access command having incomplete address information specifies a complete address for only some, but not all, of the two or more memory modules 108 being accessed. In this case, the near memory address generation unit can generate a near memory address for the memory module for which the command does not provide a complete address.

[0056] A memory access command having incomplete address information specifies a dummy address, such as a dummy address that conforms to an existing memory interface specification or standard. In this case, the near memory address generation unit can generate a near memory address for all the memory modules being accessed by the command. Such a command may be considered to have the same function as a memory access command that does not specify address information.

[0057] Memory access commands with incomplete address information involving near-memory address generation can overcome the bandwidth limitations of the command / address bus 106 for irregular and sparse computational workloads, including machine learning and large-scale graph analysis workloads that exhibit irregularities in memory access, such as accessing data in the memory module 108 at memory locations that are not address-aligned.

[0058] (PIM Support) Memory access commands with incomplete address information involving near-memory address generation can support PIM solutions (e.g., efficiently support). For example, when the same PIM operation (e.g., multiplication or addition) is applied to data stored across two or more of the memory modules 108, even if some or all of the memory locations where the data is stored are not address-aligned across two or more memory modules, a single memory access command specifying the PIM operation can be sent by the memory controller 102 via the command / address bus 106 to access the data and achieve the operation in each of the two or more memory modules. A single memory access command can be sent by the memory controller 102 via the command / address bus 106, as opposed to the memory controller 102 having to send multiple memory access commands via the command / address bus 106 to access the data and achieve the PIM operation in each of two or more memory modules. By doing so across multiple memory accesses having PIM operations applied in parallel to two or more of the memory modules 108, the bandwidth bottleneck of the command / address bus 106 can be alleviated for a particular computational workload. Further, if no address information is specified within the command or only partial address information is specified, fewer bits are sent via the command / address bus 106, so power consumption can be saved when the memory controller 102 sends a command via the command / address bus 106 compared to when the memory controller 102 sends a memory address command specifying complete address information via the command / address bus 106.

[0059] Near-memory PIM solutions can support (e.g., efficiently support) irregular and sparse memory accesses as seen in certain computational workloads, including but not limited to recommendation-based artificial neural network computations, large-scale graph analysis, sparse computations in machine learning, etc. The PIM solution places the computational logic near the memory to process lightweight and data-intensive computations, reducing the host's data bus bandwidth requirements and freeing up the host to focus on compute-intensive computations such as matrix operations.

[0060] However, as described above, PIM solutions can be limited by shared command (address) buses shared by memory modules, such as shared command / address bus 106 shared by memory module 108. For example, shared command / address bus 106 may be a shared command / address bus between banks within a channel of a DRAM memory (e.g., synchronous dynamic random access memory (SDRAM), graphics double data rate synchronous dynamic random access memory (GDDR), high bandwidth memory (HBM), etc.), but is not limited thereto. In this example, the banks are represented by memory module 108 of FIG. 1.

[0061] Many factors can limit the effectiveness of PIM solutions. As an example, the bandwidth of shared command / address bus 106 can be designed for non-PIM solution implementations and more regular computational workloads that mostly involve address-aligned memory accesses. Thus, while PIM solutions can reduce bottlenecks on the data bus, they can introduce bottlenecks on shared command / address bus 106 in irregular and sparse computational workloads that involve more memory accesses that are not address-aligned, especially across memory module 108.

[0062] Another factor that can limit the effectiveness of PIM solutions can be the sparse and irregular memory access of some computational workloads. In the case of a normal workload where data is address-aligned across memory module 108, a single broadcast PIM command can be sent by memory controller 102 via command / address bus 106 to access multiple address-aligned memory locations across memory module 108. FIG. 2 shows an exemplary series of memory access commands that can be sent by memory controller 102 via command / address bus 106 when the data stored at the memory locations is address-aligned across memory module 108.

[0063] In the example of FIG. 2, three memory access commands are sent by memory controller 102. Each of the three memory access commands is a broadcast command for each memory module 108 to receive. Each of the three commands instructs each memory module to perform an operation "OP-1" (e.g., load, multiply, or add) on the data stored at the module-relative address specified by the command. Since the data is address-aligned across memory module 108, broadcast commands can be used.

[0064] For example, assume that the operation "OP-1" is a PIM operation such as multiplication or addition. The first command 212 instructs the PIM units of each memory module to execute the operation "OP-1" using the operand "X-1" on the data stored at the memory location with the module-relative address "0" of the memory module. For example, assuming that the PIM operation "OP-1" is multiplication and the operand "X-1" is 5, the first command 212 causes the data stored at the memory location "0:0" of the memory module 108-0 to be multiplied by 5, the data stored at the memory location "1:0" of the memory module 108-1 to be multiplied by 5, and the data stored at the memory location "2:0" of the memory module 108-2 to be multiplied by 5. Note that since the data to be multiplied is stored at the same module-relative address (0) in each of the memory modules 108, a single broadcast command can be sent by the memory controller 102 via the command / address bus 106 to parallelize the PIM operations across the memory modules 108. Similarly, the same applies to the second broadcast command of FIG. 2 sent after the first broadcast command except for the module-relative address "1" and a different operand "X-2", and the third broadcast command sent after the second broadcast command except for the module-relative address "2" and a different operand "X-3". Therefore, in this example, since the data multiplied by the different operands "X-1", "X-2", and "X-3" is address-aligned across the memory modules 108, only three broadcast commands need to be sent by the memory controller 102 via the command / address bus 106. The memory address 214 represents the broadcast memory address using the above notation, where the asterisk "*" before the colon ":" represents all the memory modules 108 sharing the command / address bus 106, and the "1" after the colon represents the module-relative address.

[0065] Using broadcast memory access commands to limit the use of the common command / address bus 106 is possible when data is address-aligned across the memory module 108, but such use may not be possible when the data is not address-aligned. FIG. 3 shows an exemplary sequence of memory access commands that can be sent by the memory controller 102 via the command / address bus 106 when the data stored at the memory locations is not address-aligned across the memory module 108. The sequence of memory access commands in FIG. 3 achieves the same calculations as the sequence of memory access commands in FIG. 2. However, in the example of FIG. 3, since the data is not address-aligned, nine memory access commands can be sent by the memory controller 102 via the common command / address bus 106, whereas only three are sent in the example of FIG. 2. In particular, in the example of FIG. 3, a separate memory access command can be sent for each different memory location. In particular, the first three commands of the sequence of commands in FIG. 3 achieve the same calculations as the first broadcast command in FIG. 2, the next three commands of the sequence of commands in FIG. 3 achieve the same calculations as the second broadcast command in FIG. 2, and the last three commands of the sequence of commands in FIG. 3 achieve the same calculations as the third broadcast command in FIG. 2, but with different, non-address-aligned memory accesses. However, since the data in the example of FIG. 3 operating at the memory locations of the memory module 108 is not address-aligned, more memory access commands can be sent by the memory controller 102 via the command / address bus 106 to achieve the same calculations as the fewer commands sent in the example of FIG. 2 for address-aligned data.

[0066] Memory access command 316 represents other memory access commands within the series of commands of FIG. 3. Similar to the series of other memory access commands, in contrast to the broadcast memory access command 212 of FIG. 3, memory access command 316 addresses a specific memory location “0” within one memory module “0”. Thus, memory address 318 represents other memory addresses specified by the series of memory access commands of FIG. 3. In particular, similar to other memory addresses specified by the series of other commands, in contrast to the broadcast memory address 214, memory address 318 addresses a specific memory location “2” of one memory module “1”.

[0067] Thus, the PIM solution may not reach the bandwidth limit of the command / address bus 106 for normal computational workloads with mostly address-aligned memory accesses, but such a limit may be reached for sparse or irregular computational workloads with many non-address-aligned memory accesses because of the large number of memory access commands that can be sent by the memory controller 106 via the command / address bus 106 to achieve the same computation.

[0068] The PIM solution is supported by near-memory address generation that enables addresses to be generated in near-memory so that, even when data access is not address-aligned, parallel memory access across multiple memory modules 108 can be achieved using broadcast or multicast memory access commands.

[0069] (Memory module) FIG. 4 is a schematic diagram of an exemplary memory module that does not have a near-memory address generation function. Memory module 408 can include a memory structure 420 that provides several memory locations, with memory location 410 being an example. Memory module 408 may be electrically connected to memory controller 402 via a command / address bus. For example, memory module 408 may be any one of the memory modules 108 in FIG. 1 that are connected to memory controller 102 via command / address bus 106. Thus, memory controller 402 can be, for example, memory controller 102 in FIG. 1.

[0070] In this example, memory structure 420 has a grid or table configuration in which memory locations are arranged in rows and columns. However, memory structure 420 can be implemented as an array of memory locations. Alternatively, memory structure 420 may be a three-dimensional structure such that grids or tables of memory structures are stacked on top of each other. Memory structure 420 is not limited to a specific arrangement of memory locations, and substantially any memory structure having addressable memory locations can be used.

[0071] Also, memory module 408 includes a PIM execution unit 422 for performing near-memory calculations on data accesses from the memory cells of memory structure 420. For example, PIM execution unit 422 may include one or more multiply-accumulate units, one or more reducers for matrix-vector multiplication operations, or other sub-units for operating on data accessed from memory structure 420.

[0072] During operation, the memory controller 402 can send a memory access command via the command / address bus 106 to receive at the memory module. The memory access command can specify an address. The specified address can identify a memory location (e.g., 410) of the memory structure 420 of the memory module 408. For example, the specified address can have a row address component input to the row decoder 424 and a column address component input to the column address decoder 426. The combination of the row address and the column address specifies the memory location (e.g., 410) to be accessed. In a load (or read) operation, the data at the memory location is sent to the PIM execution unit 422 and / or returned to the host (e.g., the memory controller 402) for calculations based thereon. Each memory location of the memory structure 420 can store one or more bytes of data.

[0073] The memory module 408 can include other components not shown in FIG. 4 for simplicity, such as a row buffer and an input / output driver.

[0074] (Memory module having a near address generation unit) FIG. 5 is a schematic diagram of an exemplary memory module 508 that includes a near memory address generation unit 528 for generating a near memory address for a memory access command that specifies incomplete address information. Similar to the memory module 408 of FIG. 4, the memory module 508 can include a memory structure 520 that provides several memory locations, of which the memory location 510 is an example. The memory module 508 may be electrically connected to the memory controller 502 via a command / address bus. For example, the memory module 508 may be any one of the memory modules 108 of FIG. 1 that are connected to the memory controller 102 via the command / address bus 106. Thus, the memory controller 502 can be, for example, the memory controller 102 of FIG. 1. Similar to the memory module 408 of FIG. 4, the memory module 508 can include a PIM execution unit 522, a row decoder 524, and a column decoder 526.

[0075] However, unlike the memory module 408 of FIG. 4, the memory module 508 can include a near memory address generation unit 528 for generating a near memory address for a memory access command that specifies incomplete address information. Inside the near memory address generation unit 528, there are a row address register file (or simply "RARF" for brevity) 530 and a column address register file (or simply "CARF" for brevity) 532. The RARF 530 can store one or more row addresses for input to the row decoder 524 when selected to be generated by the row selector 534. Similarly, the CARF 532 can store one or more column addresses for input to the column decoder 526 when selected to be generated by the column selector 536. Alternatively, it may be beneficial to incorporate the near memory address generation unit into a separate (but physically proximate) integrated circuit from the memory module 508, as this allows the use of conventional memory modules and potentially provides beneficial cost savings.

[0076] The row selector 534 and the column selector 536 can control the memory access commands transmitted by the memory controller 502 via the command / address bus. In particular, the row selector 534 and the column selector 536 enable the memory controller 502 to transmit memory access commands having incomplete address information where the row address and / or the column address are respectively generated from the RARF 530 and / or the CARF 532. Also, by the row selector 534 and the column selector 536, the memory controller 502 can transmit a memory access command in which address information is not generated in the near memory by the near memory address generation unit 582.

[0077] For example, the memory controller 502 can transmit a memory access command specifying an address including both a row address and a column address. In this case, the row selector 534 may be configured to receive the row address specified in the memory access command as an input and not to receive the row address generated from the RARF 530 as an input for that command. Similarly, the column decoder 526 may be configured to receive the column address specified in the memory access command as an input and not to receive the column address generated from the CARF 532 as an input for the command. Alternatively, in the context of this example, as in the case of a DRAM, in a phase change memory, a resistive memory, or other memory structures that organize memory locations into rows and columns, the memory controller 502 can transmit a memory access command specifying a row address and subsequently transmit another memory access command specifying a column address.

[0078] As another example, the memory controller 502 can send a memory access command having incomplete address information that specifies neither a row address nor a column address. In this case, the row selector 534 may be configured to receive, as an input, the row address generated from the RARF 530 for the command, and the column selector 536 may be configured to receive, as an input, the column address generated from the CARF 532 for the command.

[0079] As yet another example, the memory controller 502 can send a first memory access command having incomplete address information that does not specify a row address, and subsequently send a second memory access command having incomplete address information that does not specify a column address. In this case, the row selector 534 may be configured to receive, as an input, the row address generated from the RARF 530 for the first command, and the column selector 536 may be configured to receive, as an input, the column address generated from the CARF 532 for the second command.

[0080] As yet another example, the memory controller 502 can send a first memory access command having incomplete address information that does not specify a row address, and subsequently send a second memory access command that specifies a column address. In this case, the row selector 534 receives, as an input, the row address generated from the RARF 530 for the first command, the column selector 536 receives, as an input, the column address specified in the second command, and may be configured not to receive, as an input, the column address generated from the CARF 532 for the second command.

[0081] As yet another example, the memory controller 502 can transmit a first memory access command specifying a row address and subsequently transmit a second memory access command having incomplete address information that does not specify a column address. In this case, the row selector 534 is configured to receive, as an input, the row address specified by the first command and not to receive, as an input, the row address generated from the RARF 530 for the first command, and the column selector 536 may be configured to receive, as an input, the column address generated from the CARF 532 for the second command.

[0082] As yet another example, the memory controller 502 can transmit a memory access command having incomplete address information that specifies one of a row address or a column address but not the other. In this case, one of the row selector 534 or the column selector 536 may be configured to receive, as an input, the row address or the column address from the RARF 530 or the CARF 532 for the command, respectively, and the other of the row selector 534 or the column selector 536 may be configured to receive, as an input, the row address or the column address specified in the memory access command, respectively.

[0083] (Program address command) The row address and / or the column address are pre-programmed into the RARF 530 and / or the CARF 532, respectively, by one or more program address commands transmitted by the memory controller 502 via the command / address bus. Subsequently, the pre-programmed address can be generated in the near memory by the near memory address generation unit 528 for one or more subsequent memory access commands having incomplete address information transmitted by the memory controller 502.

[0084] For example, the memory controller 502 can send a program address command specifying an address including both a row address and a column address to the memory module 508 via the command / address bus. The row address can be stored in the RARF 530, and the column address can be stored in the CARF 532. Then, one or more subsequent memory access commands having incomplete address information can be sent to the memory module 508 by the memory controller 502 via the command / address bus. The pre-programmed row address and the pre-programmed column address can be generated from the RARF 530 and the CARF 532 to the near memory for each of these subsequent memory access commands having incomplete address information.

[0085] As another example, the memory controller 502 can send a program address command specifying a row address or a column address, but not both a row address and a column, to the memory module 508 via the command / address bus. If a row address is specified, it can be stored in the RARF 530. On the other hand, if a column address is specified, it can be stored in the CARF 532. Then, one or more subsequent memory access commands having incomplete address information can be sent to the memory module 508 by the memory controller 502 via the command / address bus. The pre-programmed row address or the pre-programmed column address can be generated from the RARF 530 or the CARF 532 to the near memory for each of these subsequent memory access commands having incomplete address information. The subsequent memory access commands can specify the other of the row address or the column address generated in the pre-programmed near memory.

[0086] RARF530 and / or CARF532 includes a plurality of memory locations that can store a plurality of pre-programmed addresses. Each such memory location may be associated with an index (e.g., 0, 1, 2, etc.). The pre-programmed addresses may be referenced by that index within a memory access command having incomplete address information.

[0087] For example, a program address command may be sent by the memory controller 502 to pre-program the row address at index 1 of RARF530 and to pre-program the column address at index 1 of CARF532. Next, a subsequent memory access command sent by the memory controller 502 may specify index 1, and the pre-programmed row address and column address stored at index 1 of each of RARF530 and CARF532 may be generated in near memory for the command.

[0088] The memory access command can specify different indexes. For example, the memory access command may specify an index for RARF530 and a different index for CARF532. In this case, the pre-programmed row address at the index specified for RARF530 may be generated in near memory for the command, and the pre-programmed column address at the different index specified for CARF532 may be generated in near memory for the command.

[0089] The memory access command may specify only the index of RARF530 or only the index of CARF532. In this case, the pre-programmed row address or the pre-programmed column address at the RARF530 or CARF532 index specified by the command may be generated in near memory for the command, and the other of the row address and the column address may be specified by the command.

[0090] As shown, when RARF530 and / or CARF532 are pre-programmed or otherwise store an address, subsequent memory access commands having incomplete address information can be sent by the memory controller 502 via the command / address bus without the need to specify the address stored in each such subsequent memory access command. As a result, subsequent memory access commands can send fewer bits in each such command, consuming no command / address bus bandwidth that would be consumed if the stored address were specified within the memory access command.

[0091] Different memory modules (e.g., 108-0, 108-1, 108-2) are each pre-programmed or otherwise store different row and / or column addresses in their respective RARFs and / or CARFs. Thereby, a single subsequent broadcast memory access command having incomplete address information can be sent by the memory controller 502 via the command / address bus and, by leveraging the near-memory generated row and / or column addresses stored in each respective RARF and / or CARF, access data in each of the different memory modules where the data is not address-aligned across the memory modules (e.g., stored at different row and / or column addresses).

[0092] The pre-programming of the row and / or column addresses in the RARF 530 and CARF 532 by the program address commands transmitted by the memory controller 502 is executed during the idle cycles on the command / address bus. Next, the pre-programmed row and / or column addresses are repeatedly generated from the RARF 530 and / or CARF 532 to the near memory for a plurality of subsequent memory access commands transmitted by the memory controller 502. In other words, due to the near memory address generation capability provided by the RARF 530 and / or CARF 532, the bandwidth of the command / address bus consumed to transmit the program address commands via the command / address bus can be amortized over a plurality of subsequent memory access commands.

[0093] (Address calculation unit) The near memory address generation unit includes an address calculation unit for providing a near memory address calculation function to the memory module. FIG. 6 is a schematic diagram of an exemplary memory module 608 including a near memory address generation unit 628 for generating a near memory address for a memory access command that specifies incomplete address information. Similar to the memory module 508 of FIG. 5, the memory module 608 can include a memory structure 620 that provides several memory locations, where the memory location 610 is an example. The memory module 608 may be electrically connected to the memory controller 602 via a command / address bus. For example, the memory module 608 may be any one of the memory modules 108 of FIG. 1 connected to the memory controller 102 via the command / address bus 106. Accordingly, the memory controller 602 can be, for example, the memory controller 102 of FIG. 1. The memory module 608 may include a PIM execution unit 622, a row decoder 624, a column decoder 626, an RARF 630, a CARF 632, a row selector 634, and a column selector 646, similar to the memory module 508 of FIG. 5.

[0094] However, unlike the memory module 508 of FIG. 5, the memory module 608 includes an address calculation unit 638 electrically connected to the RARF 630, the CARF 632, and the PIM unit 622.

[0095] The address calculation unit 638 is configured to increment or decrement the row and / or column addresses stored in the RARF 630 and the CARF 632, respectively, using a configurable stride. The stride may be preconfigured within the address calculation unit 638. Additionally, or alternatively, the stride may be preprogrammed into the address calculation unit 638 by a memory access command having incomplete address information transmitted from the memory controller 602 via a command / address bus on which the stride is specified by a command.

[0096] For example, after a memory location in the memory structure 620 has been accessed for a memory access command having incomplete address information received by the memory module 608, the address calculation unit 638 can increment, decrement, or multiply the column address stored in the CARF 632. Similarly, in addition to, or alternatively to, this, after a memory location in the memory structure 520 has been accessed for a memory access command, the address calculation unit 638 can increment, decrement, or multiply the row address stored in the RARF 630. The next memory access command having incomplete address information received by the memory module 608 can generate an incremented, decremented, or multiplied address from the RARF 630 and / or the CARF 632. This process can continue over several memory access commands, avoiding the need to consume the bandwidth of the command / address bus by transmitting a plurality of program address commands. Instead, the second and subsequent addresses can be calculated by the address calculation unit 628 using a single start address specified by the initial program address command and a series of simple calculations by the address calculation unit 628 using a stride. The address calculation unit 628 can support a range of address calculations, including decrement by stride, multiplication by stride, etc., in addition to increment by stride.

[0097] The address calculation unit 628 can access or provide a value calculated by the PIM execution unit 622 associated with the memory structure 620, or a value accessed from the memory structure 620 to be used as an address. The address calculation unit 628 can then use such an address to calculate the row and column addresses programmed into the RARF 630 and CARF 632. Examples of such use can include reading an address offset value from the memory structure 620, adding that address offset value to a predetermined base address of a data structure using the PIM execution unit 622, and generating the row and column addresses obtained as a result programmed into the RARF 630 and CARF 632 using the address calculation unit 628.

[0098] The addition of an offset read from the memory structure 620 to the base address stored in the RARF 630 or CARF 632 may be performed by the address calculation unit 628 instead of the PIM unit 622.

[0099] Address calculation units within multiple memory modules allow a single memory access command with an incomplete address to be sent by a memory controller or host via a command / address bus and applied as an offset to different addresses within each memory module. Optionally, the base addresses associated with each memory module may differ since they are stored separately in their respective RARF / CARF and / or their respective memory structures. The address calculation unit further reduces the bandwidth requirement on the command / address bus when the offset is read from the memory structure associated with each memory module. Thus, the offset need not be sent via the command / address bus. Note that reading the offset from the memory structure can also be performed via a broadcast or multicast memory access command sent to multiple memory modules, further reducing the need for command / address bus bandwidth.

[0100] (Exemplary set of commands) Figure 7 shows an exemplary set of program address commands 740-1, 740-2, 740-3 for accessing data stored at memory locations across the unaligned memory module 108, and memory access commands 742-1, 742-2, 742-3. In this example, there are three memory access commands 742-1, 742-2, 742-3 for accessing different memory locations within the unaligned memory module 108. The start address for near memory address generation is pre-programmed in the memory module 108 by the program address commands 740-1, 740-2, 740-3 used by the first memory access command 742-1. Next, the start address is incremented by a fixed stride (e.g., 1) for the second memory access command 742-2. Thereafter, these addresses are incremented again by the fixed stride for the third memory access command 742-3. In this way, using the near memory address generation function of the memory module 108, the calculation can be achieved with only six commands transmitted by the memory controller 102 via the command / address bus 106, as opposed to the nine commands required in FIG. 3 described above.

[0101] Specifically, the program address command 740-1 is transmitted by the memory controller 102 to the memory module 108-0 via the command / address bus 106 to set the module-relative start address of module 108-0 to "0".

[0102] Next, the program address command 740-2 is transmitted by the memory controller 102 to the memory module 108-1 via the command / address bus 106 to set the module-relative start address of module 108-1 to "1".

[0103] Next, the program address command 740-2 is sent by the memory controller 102 to the memory module 108-2 via the command / address bus 106 to set the module relative start address of the module 108-2 to "2".

[0104] Then, the memory controller 102 broadcasts a memory access command 742-1 having incomplete address information (e.g., address less) to each of the memory modules 108-0, 108-1, 108-2 via the command / address bus 106. The command 742-1 specifies the operation OP-1 and the operand X-1. For example, the operation OP-1 can be a PIM command such as multiplication or addition. Although the operation OP-1 can be a PIM command, the operation OP-1 can also be a simple memory load command or other command. For example, assuming that the operation OP-1 is multiplication and the operand X-1 is the number 5, based on the module relative start address pre-programmed by the program address command 740, the command 742-1 multiplies the data stored at the memory location "0:0" of the module 108-0 by 5, multiplies the data stored at the memory location "1:1" of the module 108-1 by 5, and multiplies the data stored at the memory location "2:2" of the memory module 108-2 by 5. After each multiplication, the module relative start address of the memory module 108 can be incremented by a fixed stride (e.g., 1) such that the module relative start address becomes "1", "2", "3" for the memory modules 108-0, 108-1, 108-2, respectively.

[0105] Then, the memory controller 102 broadcasts a memory access command 742-2 having incomplete address information (e.g., address less) to each of the memory modules 108-0, 108-1, 108-2 via the command / address bus 106. The command 742-2 specifies an operation OP-1 and a different operand X-2. For example, assuming that the operation OP-1 is multiplication and the operand X-2 is the number 8, based on the current module-relative start address, the command 742-2 multiplies the data stored at the memory location "0:1" of module 108-0 by 8, multiplies the data stored at the memory location "1:2" of module 108-1 by 8, and multiplies the data stored at the memory location "2:3" of the memory module 108-2 by 8. After each multiplication, the module-relative start address of the memory module 108 can be incremented by a fixed stride (e.g., 1) such that the module-relative start address becomes "2", "3", "0" for the memory modules 108-0, 108-1, 108-2, respectively.

[0106] Then, the memory controller 102 broadcasts a memory access command 742-3 having incomplete address information (e.g., address less) to each of the memory modules 108-0, 108-1, 108-2 via the command / address bus 106. The command 742-3 specifies an operation OP-1 and a different operand X-3. For example, assuming that the operation OP-1 is multiplication and the operand X-3 is the number 0, based on the current module-relative start address, the command 742-3 multiplies the data stored at the memory location "0:2" of module 108-0 by 0, multiplies the data stored at the memory location "1:3" of module 108-1 by 0, and multiplies the data stored at the memory location "2:0" of the memory module 108-2 by 0.

[0107] (Exemplary command format) As an example, the memory controller 102 and the memory module 108 can support the following commands to support near memory address generation: memory_access_command operation,[mask] program_[row,col]_address_command memory_module_id, module_relative_address The first command described above represents an exemplary broadcast or multicast memory access command format having incomplete address information for transmission to two or more memory modules sharing a command / address bus, such as DRAM banks within a channel of an HBM memory. In the case of a broadcast command, the command can be for all memory modules sharing the command / address bus. In the case of a multicast command, the command specifies a bit mask as a parameter for selecting two or more subsets out of all memory modules sharing the command / address bus to which the command is directed. Alternatively, the command can have a mask that prohibits the command in a subset of all memory modules.

[0108] The broadcast or multicast memory access command can also specify an operation type as a parameter. This operation can be, for example, a memory load. However, if the memory module has a PIM function, a PIM operation such as multiplication or addition for near-memory processing can be specified. Note that, unlike conventional memory access commands, the above memory access commands can be transmitted without specifying address information or only incomplete address information. Also, unlike conventional memory access commands, the near-memory address generation unit of the memory module can generate a missing near-memory address or a missing address portion of the memory for the above memory access commands.

[0109] The second command is an exemplary program address command format for transmission to a memory module via a shared command / address bus. The command accepts, as a parameter, an identifier of the memory module that is the target of the command. The command also accepts an address parameter that can be a row address, a column address, a row and column address, or other module-relative address that can vary depending on the type of program address command. For example, there can be different types of program address commands for separately programming RARF and CARF with a row address and a column address, respectively.

[0110] The program address commands presented herein program each memory module separately. Alternatively, a broadcast program address command or a multicast program address command with an associated mask can program multiple memory modules having the same module-relative address. Also, as described above, further variations of the program address command can separately include programming of row and column addresses or other module-relative addresses.

[0111] (Other Considerations) Certain computer memory technologies, such as hybrid memory cubes (or simply "HMC" for brevity) and HBM memory, can stack multiple memory layers on a base die. Such base dies can potentially accommodate near-memory computing functions, such as the near-memory address generation units 528 and 628 of FIGS. 5 and 6, respectively.

[0112] The near-memory address generation units 528 and 628 can be placed near the memory arrays / banks within the memory layers of a three-dimensional (3-D) stacked memory or a conventional DRAM.

[0113] Embodiments are also applicable to other forms of memory other than DRAM, such as non-volatile memory, flash memory, and embedded DRAM.

[0114] While some embodiments show state and logic for associating a single starting address with any of the proposed memory access commands, by replicating the proposed state and logic for multiple PIM operations, multiple starting addresses can be tracked by different PIM operations such as tracking different addresses within a near-memory address generation unit or each memory module for multiplication and addition operations.

[0115] The memory controller 102 can use a deterministic double data rate (or simply "DDR" for short) using the memory module 108. DDR can be supported, according to some embodiments, by properly synchronizing between the memory structures 108. For example, a memory access command that requires a DRAM row open (activation) may take longer than a DRAM row buffer hit. The memory controller 102 can be extended to synchronize row opens across different memory modules to allow broadcast or multicast memory access commands with incomplete address information to access different addresses across different memory modules.

[0116] (Conclusion) The references to "embodiments" in this specification mean that a particular feature, structure, or characteristic is described in connection with at least one embodiment of the invention and may be included in at least one embodiment of the invention. The appearances of the phrases "in an embodiment" and "according to an embodiment" in various places in this specification do not necessarily all refer to the same embodiment, nor are they another or alternative embodiment mutually exclusive with other embodiments.

[0117] Some of the various drawings show some logical stages in a particular order, but stages that are not order-dependent may be rearranged, and other stages may be combined or divided. Although some rearrangements or other groupings are specifically mentioned, the rearrangements and groupings presented herein are not an exhaustive list of options.

[0118] In the above detailed description and the appended claims, terms such as first, second, etc. are sometimes used herein to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first computing device can be referred to as a second computing device, and similarly, a second computing device can be referred to as a first computing device. The first computing device and the second computing device are both computing devices, but they are not the same computing device.

[0119] In the above detailed description, the singular forms "a", "an", and "the" are intended to include the plural as well, unless the context clearly dictates otherwise. When used in the above detailed description and the appended claims, the term "and / or" refers to any one or more of the related listed items and all possible combinations, and includes them.

[0120] When used in the appended claims and the above detailed description, the phrases "based on", "according to", "include", "including", "comprises", and / or "comprising" specify the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0121] When used in the foregoing detailed description and the appended claims, the term "if" is optional and, depending on the context, is construed to mean "when," "upon," "in response to determining," "in response to detecting," or "in accordance with a determination that." Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" are optional and, depending on the context, are construed to mean "upon determining" or "in response to determining," or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]" or "in accordance with a determination that [a stated condition or event] is detected."

[0122] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a limiting sense.

Claims

1. 1. A memory controller, comprising: Equipped with command logic, The command logic includes: sending a program address command specifying a particular address to a set of two or more memory modules over the connection; sending, via the connection, a memory access command having incomplete address information for accessing data stored in a set of two or more memory locations of the set of two or more memory modules corresponding to a set of two or more near memory generated addresses based on the particular address; The device is configured to: each of the two or more memory locations is in a different one of the two or more memory modules; Memory controller.

2. at least two near memory generated addresses of the set of two or more near memory generated addresses are different addresses; The memory controller of claim 1 .

3. the command logic is configured to send a second program address command having a second address over the connection to a second memory module of the set of two or more memory modules; transmitting the memory access command includes transmitting the memory access command to access data stored in the set of two or more memory locations based on the particular address and the second address. The memory controller of claim 1 .

4. A particular operation to be performed by a set of two or more processor-in-memory units of the set of two or more memory modules on data stored in the set of two or more memory locations is specified by the memory access command. The memory controller of claim 1 .

5. the incomplete address information includes either (a) a row address common to the set of two or more memory locations; or (b) a column address common to the set of two or more memory locations. The memory controller of claim 1 .

6. the command logic is configured to transmit, over the connection, the memory access command including a mask that prohibits the memory access command in a subset of a plurality of memory modules that share the connection. The memory controller of claim 1 .

7. the command logic is configured to transmit, over the connection, a plurality of memory access commands for accessing data stored in a set of two or more memory locations of the set of two or more memory modules corresponding to a set of two or more near memory generated addresses. The memory controller of claim 1 .

8. the command logic is configured to transmit, over the connection, the memory access command specifying a register file index for accessing data stored in the set of two or more memory locations; the register file index is usable in the set of two or more memory modules to generate the set of two or more near memory generated addresses; The memory controller of claim 1 .

9. 1. A memory module comprising: a memory structure having a plurality of memory locations; a near memory address generation unit; The near memory address generation unit includes: configured to generate addresses corresponding to memory locations among the plurality of memory locations based on receiving at the memory module via a connection a program address command specifying a particular address for a set of two or more memory modules, and a memory access command having incomplete address information for accessing data stored in a set of two or more memory locations of the set of two or more memory modules based on the particular address, the memory access command corresponding to a set of two or more near memory generated addresses; each of the two or more memory locations is in a different one of the two or more memory modules; the set of two or more memory modules includes the memory module, the set of two or more memory locations includes the memory location, the set of two or more near memory generated addresses includes the address; Memory modules.

10. the near memory address generation unit includes a register file configured to store the addresses corresponding to the memory locations based on receipt at the memory module of the program address command via the connection; 10. The memory module of claim 9.

11. the address is a first address, the memory location is a first memory location; the near memory address generation unit includes a register file capable of storing a second address corresponding to a second memory location of the plurality of memory locations; The near memory address generation unit includes: incrementing the second address by a stride to calculate the first address; decrementing the second address by a stride to calculate the first address; or and multiplying the second address by a stride to calculate the first address.

10. The memory module of claim 9.

12. a processor in-memory execution unit; the near-memory address generation unit includes an address calculation unit configured to generate the address corresponding to the memory location based on a value calculated by the processor in-memory execution unit; 10. The memory module of claim 9.

Images