Memory access commands with near-memory address generation

Abstract

A memory controller is provided that is configurable with command logic that is capable of sending memory access commands with incomplete address information via a command / address bus that connects the memory controller to a memory module. The memory controller can send memory access commands via the bus for accessing data stored at memory locations of the memory module. The memory locations can correspond to different near-memory generated addresses, reflecting that the data has not been address aligned on the memory module. However, due to near-memory address generation, the memory controller can send memory access commands with incomplete address information for accessing data stored at different addresses, as opposed to having to send multiple memory access commands on the bus that specify complete address information for accessing data at different addresses, thereby conserving usage of available bus bandwidth, reducing power consumption, and increasing compute throughput.

Description

Technical Field

[0001] Publicly disclosed implementations of memory access commands with near-memory address generation generally relate to volatile and non-volatile computer memory, and more specifically, to computer implementation logic for accessing data stored in computer memory. Background Technology

[0002] Computer memory is a fundamental component of almost every computer, including personal computers, tablets, smartphones, server computers, and other computing devices such as printers and internet-connected devices. Computer memory is typically used in computers to store data and the processor-executable instructions (machine code) that operate on that data.

[0003] While the term "memory" is typically used to refer to volatile computer memory, such as Dynamic Random Access Memory (or simply "DRAM"), which takes the form of integrated circuit chips with metal-oxide-semiconductor memory cells, computer memory can also be non-volatile. A non-exhaustive list of volatile and non-volatile computer technologies includes DRAM, Static Random Access Memory (or simply "SRAM"), Non-volatile Dual In-line Memory Modules (or simply "NVDIMM"), Flash Memory, Embedded DRAM, Notepad Memory, and others.

[0004] A computer's memory is typically connected to the computer's hardware "host" via a set of wires or conductors. The hardware host can encompass the memory controller and central processing unit (or simply "CPU"), graphics processing unit (or simply "GPU"), or other hardware data processors. The connection between the computer memory and the host typically includes an interface for command / address information (sometimes called the "command / address bus") and an interface for data information (sometimes called the "data bus").

[0005] Recently, the computing power and speed of processors (such as CPUs and GPUs) have increased faster than the bandwidth of the data bus connecting computer memory and the host. As a result, the data bus has become a bottleneck for computing throughput as processors leisurely wait for data to be transferred to and from computer memory via the data bus.

[0006] Processor-in-Memory (PIM) solutions provide data processing capabilities close to the computer's memory (e.g., on the same die). With PIM solutions, computations can be performed on data accessed from the computer's memory because the data processing capabilities are located close to the computer's memory, without having to transfer all the raw data to the host via a data bus. For example, one possible PIM solution involves adding simple vector computation elements to each internal memory module of the computer's memory, such as to each DRAM library or subarray. The host can then use these additional elements to trigger computations at all internal memory modules without transferring data over the DRAM interface via a data bus.

[0007] By transferring data between computer memory and the host, PIM solutions can save bandwidth consumption on the data bus and increase throughput for computing workloads, especially for the growing categories of data-constrained computing workloads, such as deep artificial neural network machine learning workloads, large-scale graphics analysis workloads, sparse computation in machine learning workloads, and other types of compute-intensive workloads. Without a local PIM unit, these compute-intensive workloads may reach the bandwidth limit of the data bus due to data transfer between computer memory and the host.

[0008] As just one example, a common operation in machine learning is sparse embedding lookup. This operation can dominate the execution time of some workloads. From a large embedding table containing millions of embedding entries stored in computer memory, such an operation can sparsely read large embeddings containing hundreds of elements and aggregate them. Due to limitations in data reuse, such operations are highly sensitive to data bus bandwidth.

[0009] Without a PIM solution, a sparse embedding lookup operation might require the host to issue a set of memory load requests, one for each of multiple memory modules (e.g., a DRAM library). Each memory load request causes data to be transferred from the corresponding memory module to the host via the data bus. Since memory modules can share command / address and data buses (e.g., DRAM libraries on the same memory channel), it might be necessary to serialize access to different memory modules, resulting in full utilization of both the command / address and data buses. Conversely, with a PIM solution, the accumulation can be calculated in the memory-local PIM cell without transferring some data on the data bus.

[0010] However, because PIM solutions can alleviate some or all of the data bus bottlenecks for certain workloads, bottlenecks at the command / address bus can become new bottlenecks. For example, bottlenecks at the command / address bus may arise with data-constrained computational workloads that have sparse or irregular memory access patterns (such as those found in machine learning workloads and other workloads). Since the command / address bus is shared by the memory modules, each memory load command still needs to be sent via the command / address bus.

[0011] One possible solution to the command / address bus bottleneck is to increase the bandwidth of the command / address bus by increasing the number of memory channels or by increasing the number of command / address lines per memory channel. However, this increases the unit manufacturing cost and power consumption. Furthermore, for workloads with regular memory access patterns, the additional command / address bus bandwidth may not be utilized.

[0012] The disclosed implementation plan addresses this issue and other problems.

[0013] The methods described in this section are feasible methods, but not necessarily methods that have been previously conceived or implemented. Therefore, unless otherwise stated, it should not be assumed that any method described in this section is prior art, or is known, routine, or conventional, simply because it is included in this section. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of an exemplary system for generating memory access commands with near memory addresses.

[0015] Figure 2 An exemplary series of memory access commands is shown, which can be sent by the memory controller via the command / address bus after the data stored in the memory location of the memory module has been address-aligned on the memory module.

[0016] Figure 3 An exemplary series of memory access commands is shown, which can be sent by the memory controller via the command / address bus when the data stored in the memory location of the memory module has not yet been address-aligned on the memory module.

[0017] Figure 4 An exemplary memory module without near memory address generation capability is shown.

[0018] Figure 5An exemplary memory module including a near memory address generation unit is shown, which is used to generate a near memory address for a memory access command that specifies incomplete address information.

[0019] Figure 6 An exemplary memory module including a near memory address generation unit is depicted, the near memory address generation unit having an address calculation unit for generating a near memory address for a memory access command specifying incomplete address information.

[0020] Figure 7 This demonstrates a series of commands for accessing data stored on multiple memory modules.

[0021] Not all components depicted in the accompanying drawings are essential, and embodiments may include additional components not shown in the drawings. Changes may be made to the arrangement and type of components without departing from the scope of this disclosure. Within the scope of this disclosure, additional components, different components, or fewer components may be used. Detailed Implementation

[0022] In the following description, numerous specific details are set forth for illustrative purposes to provide a thorough understanding of implementations of memory access commands with near-memory address generation. However, it will be apparent that these implementations can be practiced without these specific details. In other instances, well-known structures and apparatuses are shown in block diagram form to avoid unnecessarily obscuring these implementations.

[0023] General Overview

[0024] This invention discloses systems and methods for memory access commands with near-memory address generation. Such systems and methods facilitate computer memory access to support (e.g., efficiently support) computational workloads, including but not limited to deep machine learning workloads (e.g., using deep artificial neural networks), large-scale graphics analysis, and / or other computational workloads with sparse or irregular memory access. The subject matter can be used in a variety of systems, including but not limited to northbridge chipsets, field-programmable array (or simply "FPGA") devices, microprocessor devices, CPU devices, GPU devices, memory integrated circuits, and / or other systems such as machine learning and / or artificial intelligence accelerator devices.

[0025] A memory controller configured with command logic can send memory access commands with incomplete address information via a connection between the memory controller and two or more computer memory modules. The incomplete address information may be empty (no address). Conversely, it may contain partial address information. The memory controller sends memory access commands via the connection to access data stored at two or more memory locations within the memory modules. These memory locations correspond to two or more nearest-memory address pairs.

[0026] The near-memory address corresponding to the memory location can be different, reflecting that the data has not yet been address-aligned on the memory module. However, due to the near-memory address generation, the memory controller can send memory access commands with incomplete address information to access data stored at different addresses. This is the opposite of having to send multiple memory access commands with complete address information on the bus to access data at different addresses, thereby saving available bus bandwidth, reducing power consumption, and increasing computational throughput.

[0027] The memory controller's command logic is configured to send program address commands with specified addresses to the memory modules via connections. These pre-programmed addresses can be stored close to the memory, can vary for different memory modules, and can be balanced among multiple memory access commands with incomplete address information, thus avoiding the need to send complete address information via a connection to each memory access command.

[0028] Memory access commands with incomplete address information can specify the operation to be performed (e.g., multiplication or addition) by a PIM cell in a memory module. Due to near memory address generation, sending a single memory access command with incomplete address information may cause the PIM cell to perform operations on data stored in different memory modules, even if the data has not yet been address-aligned on the memory modules.

[0029] One of the memory modules has a memory structure and a near-memory address generation unit. The memory structure covers addressable memory locations. The near-memory address generation unit is configured to generate an address corresponding to a memory location in the memory structure, close to the memory. The near-memory address generation unit is configured to generate the address based on a memory access command with incomplete address information received at the memory module via the connection, which is sent by the memory controller via the connection, for accessing data stored at the memory location of the memory module corresponding to the near-memory address.

[0030] The near memory address generation unit includes a register file. The register file is configured to store addresses that the near memory address generation unit will generate. Based on a program address command received at the memory module via this connection, specifying the address to be stored in the register file, that address is stored in the register file.

[0031] The near-memory address generation unit includes an address calculation unit configured to calculate the address to be generated by the near-memory address generation unit. The address calculation unit calculates the address to be generated based on a pre-configured step size applied to the address. The address calculation unit calculates the address to be generated based on values ​​obtained from the PIM cells of the memory module or based on values ​​obtained from the memory location of the memory structure.

[0032] The memory controller includes command logic configured to send, via a link, a first memory access command with incomplete address information for accessing data stored in a set of two or more rows of two or more memory modules, the set of two or more rows corresponding to a set of two or more near-memory generated row addresses. The command logic is also configured to send, via a link, a second memory access command with incomplete address information for accessing data stored in a set of two or more columns of two or more memory modules, the set of two or more columns corresponding to a set of two or more near-memory generated column addresses.

[0033] The memory module includes a memory structure with multiple rows and columns. The memory module also includes a near-memory address generation unit configured to generate a row address corresponding to a row in the multiple rows based on a first memory access command received by the module via a connection for accessing data stored in a row. The near-memory address generation unit is further configured to generate a column address corresponding to a column in the multiple columns based on a second memory access command received by the module via a connection for accessing data stored in a column of that row.

[0034] When processing computational workloads via irregular or sparse data access to computer memory, the implementations support (e.g., efficiently support) CPUs, GPUs, and PIM units. Such workloads include deep learning recommendation models and large-scale graph analytics, which are increasingly prevalent in business, government, education, and research. The implementations allow for faster memory access and lower power consumption for these workloads.

[0035] These and other implementation schemes will now be described in more detail with reference to the accompanying drawings.

[0036] Systems for memory access commands with near memory address generation

[0037] Figure 1This is a schematic diagram of an exemplary system for generating memory access commands with near-memory addresses. The system includes a memory controller 102 electrically connected to a command / address bus 106 shared by two or more memory modules 108, and the memory modules 108 are also electrically connected to the command / address bus. The electrical connections can be direct or indirect, wherein indirect electrical connections may include additional structures in the electrical path.

[0038] The system may include Figure 1 Other components not shown. For example, the system may include... Figure 1 Other buses, chips, and circuits not shown.

[0039] exist Figure 1 The example depicts three memory modules. However, the system may have as few as two or more memory modules, including more than three. In addition to the command logic 104, the memory controller 102 may also include... Figure 1 Other logic and structures not depicted include request cache, response cache, memory mapping logic, and arbitration logic. In addition to being electrically connected to the memory module 108 via the command / address bus 106, the memory controller 102 can also be connected via... Figure 1 Other buses, not depicted, such as the data bus, are electrically connected to the memory module 108.

[0040] The memory controller 102 may include hard-wired digital circuitry and / or programmable digital circuitry for managing the flow of data to and from the memory module 108. The memory controller 102 may be a separate chip or integrated into another chip, such as being placed on the same die or as an integrated component of a microprocessor such as a CPU or GPU. In addition to the command logic 104, the memory controller 102 may also encompass logic for reading and writing data to the memory module 108. In the case that the memory module 108 is a volatile memory device, the memory controller 102 may include logic for periodically refreshing the charge in the memory cells of the memory module 108, such as in the case where the memory module 108 is implemented based on, for example, semiconductor DRAM.

[0041] Through the shared command / address bus 106, commands sent by the memory controller 102 to one, two, or more memory modules 108 can be transmitted via the command / address bus 106. When a command is sent from the memory controller 102 to one or more memory modules 108 via the command / address bus 106, the command can be sent from the memory controller 102 to one or more memory modules solely along the command / address bus 106, or along one or more additional electrical connections in the electrical path between the memory controller 102 and one or more memory modules.

[0042] Command / address bus 106 may encompass a group of one or more electrical wires or conductors for transmitting electrical signals representing commands, including program address commands and memory access commands with incomplete address information disclosed herein. Electrical signals may be transmitted from memory controller 102 to memory module 108 via command / address bus 106.

[0043] The connection between the memory controller 102 and the memory module 108 may encompass an interface for sending and receiving command / address information, such as memory access commands. Figure 1 In this document, this connection is labeled Command / Address Bus 106. The same or separate connection between the memory controller 102 and the memory module 108 may encompass the interface used for sending and receiving data, such as data accessed from a memory location. This connection is sometimes referred to herein as the “data bus.” Although the term “bus” is used, one or more connections may encompass a set of wires or conductors, or other forms of wired and / or radio coupling or electrical connection between the memory controller 102 and the memory module 108. Furthermore, as indicated, the Command / Address Bus 106 and the Data Bus may share the same electrical connection between the memory controller 102 and the memory module 108 or use separate electrical connections.

[0044] The command / address bus 106 can be composed of sub-buses. For example, the command / address bus 106 can be composed of a row address bus and a column address bus, which are possible other sub-buses.

[0045] Memory module 108 provides an addressable memory location for storing data. In some cases, the data stored or accessed at a memory location is referred to herein as a “value” or “data value.” Such a value can be represented by one or more bits. In some cases, the memory location is byte-addressable and stores an 8-bit data value. In some cases, the memory location is word-addressable and stores a multi-byte data value (e.g., a 16-bit, 32-bit, 64-bit, or 128-bit data value).

[0046] When a data value is accessed from a memory location (e.g., 110) of a memory module (e.g., 108-0) via a command sent by the memory controller 102, the data value can be sent from the memory module back to the host via the data bus between the memory module 108 and the host. Alternatively, the data value can be sent to a PIM unit located near the memory module for calculation or accumulation (e.g., multiplication, addition, etc.) by the PIM unit.

[0047] As used herein, the term "near memory" encompasses memory, such as in the same die, chip, or circuitry as the memory module (e.g., 108-0). However, more broadly, near memory encompasses a structure that is actually closer to the memory module than the host or memory controller 102, which is located at the other end of the command bus 106 relative to the memory module.

[0048] Memory modules 108-0, 108-1, and 108-2 are all DRAM libraries. However, each memory module 108 may correspond to another type of component in a memory hierarchy, such as a subarray, level, or channel. Furthermore, memory modules are not limited to volatile memory devices or DRAM devices, and may include non-volatile memory devices or other types of memory devices with addressable memory locations.

[0049] A near memory address generation unit is used to generate near memory addresses. For example, a separate near memory address generation unit may be located within each memory module 108-0, 108-1, and 108-2. For example, in the case where memory module 108 is a component of a stack of multiple memory layers on top of a basic die, the near memory address generation unit may be located in the basic die, or the near memory address generation unit may be located in the memory layers of a three-dimensional stacked memory or in the memory layers of a more conventional DRAM.

[0050] Besides 3D stacked DRAM and conventional DRAM, the near memory address generation unit can also be located in other forms of memory, such as non-volatile memory, flash memory, and embedded DRAM. Although the embodiments described below are in the context of DRAM memory architecture, the near memory address generation unit and its functionality can be implemented in other memory contexts, such as other emerging or later-developed memory technologies, including non-volatile memory modules such as NVDIMM, and other forms of memory such as notepad memory.

[0051] Address alignment data

[0052] The near-memory address generation unit associated with memory module 108 enables memory controller 102 to send memory access commands with incomplete address information (such as no address information or only partial address information). Nevertheless, the near-memory address generation unit can generate a near-memory address to provide complete address information, which is either complete or combined with the partial address information specified in the memory access command. This supports computational workloads such as machine learning and large-scale graphics analysis, as well as other computational workloads that generate irregular memory accesses, while reducing the bandwidth consumption of the command / address bus 106 and lowering power consumption.

[0053] For example, since the near memory address generation unit is associated with memory module 108, a memory access command with incomplete address information can be sent from memory controller 102 to two or more memory modules in memory module 108 via command / address bus 106. This allows access to memory locations in different memory modules within memory module 108 even if the memory locations are not yet address aligned. For example, memory location 110 of memory module 108-0 is located in... Figure 1 The colon indicates that the address is "0:2". This note is for illustrative purposes only. The number before the colon ':' addresses the memory module in memory module 108 connected to command 106 (e.g., an identifier) ​​("0" in this example). The number after the colon addresses the memory location within the addressed memory module ("2" in this example). In some cases, this part of the address that addresses the specific memory location within the memory module addressed by the colon is referred to herein as a "module-relative" address. In this example, the "2" after the colon will be the module-relative address of address "0:2".

[0054] Furthermore, it should be emphasized that in this disclosure, Figure 1 The address comments are for illustrative purposes only. An actual address can be a multi-bit address with multiple components, such as a number of bits addressing a memory module, a number of bits addressing the row address of the addressed memory module, a number of bits addressing the column address of the addressed memory module, and other information represented by the bits. Other multi-bit address representations are also possible, and no specific address representation is required.

[0055] Some computing workloads exhibit regular memory access patterns where memory accesses are address-aligned across memory module 108. For example, memory location 110 at address "0:2" of memory module 108-0 is address-aligned with memory location 110 at address "1:2" of memory module 108-1, and memory location 110 at address "1:2" of memory module 108-1 is address-aligned with memory location 110 at address "2:2" of memory module 108-2, because all three memory locations have the same module-relative address ("2" in this example).

[0056] In cases where data to be accessed from two or more memory modules is address-aligned, a single memory access command with complete address information can be sent to access multiple address-aligned memory locations. For example, memory controller 102 can send (e.g., broadcast) a single memory access command specifying "2" as a module-relative address and accessing data stored at memory locations "0:2", "1:2", and "2:2". For example, a single memory access command can be broadcast by memory controller 102 on command / address bus 106 to all memory modules 108 specifying "2" as a module-relative address, which explicitly identifies the memory location to be accessed at each memory module 108.

[0057] However, if the data to be accessed in two or more memory modules 108 is not yet address aligned, it may be necessary for the memory controller 102 to send multiple memory access commands. For example, if the data to be accessed is stored in memory locations “0:2” and “1:3”, two memory access commands specifying different addresses may need to be sent by the memory controller 102 via the command / address bus 106.

[0058] However, since the near memory address generation unit is associated with memory module 108, the memory controller 102 can send a single memory access command with incomplete address information via command / address bus 106 to access data stored at different unaligned memory locations on two or more memory modules in memory module 108. For example, the command can access data stored at memory locations “0:2”, “1:3”, and “2:0”, some or all of which can be generated by the near memory address generation unit associated with memory module 108. For example, based on a command received from memory controller 102 via command bus 106 from memory module 108-0, the first near memory address generation unit associated with memory module 108-0 can generate the module relative address “2”. Based on a command received from memory controller 102 via command bus 106 from memory module 108-1, the second near memory address generation unit associated with memory module 108-1 can generate the module relative address “3”. Based on the memory module 108-2 receiving commands from the memory controller 102 via the command bus 106, the third near memory address generation unit associated with the memory module 108-2 can generate the module relative address "0".

[0059] Incomplete address information

[0060] As used herein, a memory access command with “incomplete” address information encompasses a command sent by memory controller 102 via command / address bus 106 to two or more memory modules 108, wherein some or all of the module relative addresses are generated close to the memory. As mentioned above, the module relative address may encompass a portion of a complete address that identifies the memory location of the memory module addressed by the complete address to be accessed by the command. Conversely, a memory access command with “complete” address information may encompass a command sent by memory controller 102 via command / address bus 106 to two or more memory modules 108, wherein the entire module relative address may be specified by the command, and no portion of the module relative address may be generated close to the memory.

[0061] Memory access commands with incomplete address information may not specify any part of the module relative address. In this case, the entire module relative address can be generated closer to the memory. For example, a memory access command with incomplete address information may not specify either the row address or the column address. In this case, the row address and column address can be generated closer to the memory by the proximity memory address generation unit. Note that due to the proximity memory address generation unit, different row addresses and column addresses can be generated closer to the memory for different memory modules being accessed, such as when the data has not yet been address-aligned on the memory modules. Furthermore, when the data has not yet been address-aligned between the memory modules to be accessed, due to the proximity memory address generation unit, the memory controller 102 only needs to send a single memory access command via the command / address bus 106, and does not need to send a separate memory access command for each different module relative address, thereby saving bandwidth and power consumption of the command / address bus 106.

[0062] Memory access commands with incomplete address information can specify only partial address information, such as only some, but not all, module-relative addresses. For example, a memory access command with incomplete address information can only specify the row address or column address shared by the memory modules being accessed, but cannot specify both row and column addresses, or specify complete addresses, or specify all module-relative addresses. In this case, the missing portion of the module-relative address can be generated closer to the memory.

[0063] For example, a memory access command with incomplete address information can specify the row address for the memory module being accessed, but not the column address. In this case, the column address can be generated close to the memory by the near-memory address generation unit. Note that due to the near-memory address generation unit, different column addresses can be generated close to the memory for different memory modules being accessed, such as when the data has not yet been address-aligned on the memory modules. Furthermore, when the data has not yet been address-aligned between the memory modules to be accessed, due to the near-memory address generation unit, the memory controller 102 only needs to send a single memory access command via the command / address bus 106, and does not need to send a separate memory access command for each different column address, thereby saving bandwidth and power consumption of the command / address bus 106.

[0064] As another example, a memory access command with incomplete address information can specify a column address command for the memory module being accessed, but not a row address. In this case, the row address can be generated close to the memory by the near-memory address generation unit. Note that due to the near-memory address generation unit, different row addresses can be generated close to the memory for different memory modules being accessed, such as when the data has not yet been address-aligned on the memory modules. Furthermore, when the data has not yet been address-aligned between the memory modules to be accessed, due to the near-memory address generation unit, the memory controller 102 only needs to send a single memory access command via the command / address bus 106, and does not need to send a separate memory access command for each different row address, thereby saving bandwidth and power consumption of the command / address bus 106.

[0065] For some, but not all, of the two or more memory modules 108 being accessed, a memory access command with incomplete address information specifies a complete address. In this case, for the memory module whose command does not provide a complete address, the nearest memory address generation unit can generate an address closer to the memory.

[0066] Memory access commands with incomplete address information specify pseudo-addresses, such as pseudo-addresses compatible with existing memory interface specifications or standards. In this case, the nearest memory address generation unit can generate an address closer to the memory for all memory modules being accessed by the command. Such commands can be considered functionally equivalent to memory access commands that do not specify any address information.

[0067] Memory access commands with incomplete address information generated from near-memory addresses can overcome the bandwidth limitations of the command / address bus 106 for irregular and sparse computational workloads, including machine learning and large-scale graphics analysis workloads, which exhibit irregularities in memory access, such as accessing data in memory modules 108 at memory locations that have not yet been address aligned.

[0068] PIM support

[0069] Memory access commands with incomplete address information generated from near-memory addresses can support (e.g., effectively support) PIM solutions. For example, in cases where the same PIM operation (e.g., multiplication or addition) is to be applied to data stored on two or more memory modules 108, a single memory access command specifying the PIM operation can be sent by the memory controller 102 via the command / address bus 106 to access the data and complete the operation at each of the two or more memory modules, even if some or all of the memory locations storing the data are not address-aligned across the two or more memory modules. Instead of the memory controller 102 having to send multiple memory access commands via the command / address bus 106 to access the data and complete the PIM operation at each of the two or more memory modules, a single memory access command can be sent by the memory controller 102 via the command / address bus 106. Thus, by having multiple memory accesses that apply PIM operations in parallel to two or more memory modules 108, the bandwidth bottleneck of the command / address bus 106 for certain computational workloads can be alleviated. Furthermore, when no address information is specified in the command or only partial address information is specified, the memory controller 102 can save power when sending the command via the command / address bus 106 compared to when the memory controller 102 sends the memory address command with full address information via the command / address bus 106, because fewer bits are sent via the command / address bus 106.

[0070] Near-Memory (PIM) solutions can support (e.g., effectively support) irregular and sparse memory accesses, such as those found in certain computational workloads, including but not limited to recommendation-based artificial neural network computations, large-scale graph analysis, and sparse computations in machine learning. PIM solutions place computational logic close to memory to handle lightweight, data-intensive computations, thereby reducing the host's data bus bandwidth requirements and freeing up the host to focus on computationally intensive computations such as matrix mathematics.

[0071] However, as mentioned, the PIM solution may be limited by a shared command (address) bus shared by memory modules, such as the shared command / address bus 106 shared by memory module 108. For example, the shared command / address bus 106 may be, but is not limited to, a shared command / address bus between libraries within a DRAM memory channel (e.g., Synchronous Dynamic Random Access Memory (SDRAM), Graphics Dual Data Rate Synchronous Dynamic Random Access Memory (GDDR), High Bandwidth Memory (HBM), etc.). In this example, these libraries are... Figure 1 The memory module 108 is used to represent this.

[0072] Several factors can limit the effectiveness of a PIM solution. Firstly, the bandwidth of the shared command / address bus 106 can be designed for non-PIM solution implementations and more regular computational workloads, primarily encompassing address-aligned memory accesses. Therefore, while a PIM solution may alleviate bottlenecks on the data bus, it may introduce bottlenecks onto the shared command / address bus 106, especially for irregular and sparse computational workloads that encompass further memory accesses not yet address-aligned on memory module 108.

[0073] Another factor that may limit the effectiveness of a PIM solution could be the sparse and irregular memory accesses of some computational workloads. For a regular workload in which data is address-aligned on memory module 108, a single broadcast PIM command can be sent by memory controller 102 via command / address bus 106 to access multiple address-aligned memory locations on memory module 108. Figure 2 An exemplary series of memory access commands is shown, which can be sent by the memory controller 102 via the command / address bus 106 after the data stored in the memory location has been address-aligned on the memory module 108.

[0074] exist Figure 2 In the example, memory controller 102 sends three memory access commands. All three memory access commands are broadcast commands to be received by each memory module in memory module 108. Each command instructs each memory module to perform an operation "OP-1" (e.g., load, multiply, or add) on the data stored at the module-relative address specified by the command. Broadcast commands are used because the data is address-aligned on memory module 108.

[0075] For example, suppose operation “OP-1” is a PIM operation such as multiplication or addition. First command 212 instructs the PIM unit at each memory module to perform operation “OP-1” on the data stored at the memory location at the module-relative address “0” of the memory module using operand “X-1”. For example, suppose PIM operation “OP-1” is multiplication and operand “X-1” is 5, then first command 212 can multiply the data stored at memory location “0:0” of memory module 108-0 by 5, multiply the data stored at memory location “1:0” of memory module 108-1 by 5, and multiply the data stored at memory location “2:0” of memory module 108-2 by 5. Note that since the data to be multiplied is stored at the same module-relative address (“0”) at each memory module 108, a single broadcast command can be sent by memory controller 102 via command / address bus 106 to parallelize the PIM operations on memory modules 108. Similarly, for Figure 2 The second broadcast command, except for the module relative address being "1" and the distinct operand being "X-2", is sent after the first broadcast command, and the third broadcast command, except for the module relative address being "2" and the distinct operand being "X-3", is sent after the second broadcast command. Therefore, in this example, since the data to be multiplied by the distinct operands "X-1", "X-2", and "X-3" are address-aligned on memory module 108, only three broadcast commands need to be sent by memory controller 102 via command / address bus 106. Memory address 214 represents the broadcast memory address using the aforementioned annotation, where the asterisk '*' before the colon ':' indicates all memory modules 108 sharing command / address bus 106, and the "1" after the colon indicates the module relative address.

[0076] While it is possible to restrict the use of the shared command / address bus 106 by using broadcast memory access commands after the data has been address-aligned on memory module 108, such use may not be possible when the data has not yet been address-aligned. Figure 3 An exemplary series of memory access commands is shown, which can be sent by the memory controller 102 via the command / address bus 106 when the data stored in the memory location has not yet been address-aligned on the memory module 108. Figure 3 A series of memory access commands completed with Figure 2 A series of memory access commands are similar to computations. However, due to... Figure 3 In the example, the data is not yet address aligned, so nine memory access commands can be sent by the memory controller 102 via the shared command / address bus 106, which is consistent with... Figure 2In the example, only three opposites are sent. Specifically, in Figure 3 In the example, separate memory access commands can be sent for each different memory location. Specifically, Figure 3 The first three commands in the command series completed the task with... Figure 2 The first broadcast command is similar to the calculation. Figure 3 The last three commands in the command series completed the task with... Figure 2 The second broadcast command is calculated similarly, and Figure 3 The last three commands in the command series completed the process with... Figure 2 The third broadcast command performs a similar calculation, but involves different and non-address-aligned memory accesses. However, due to... Figure 3 In the example, the data to be operated on at the memory location of memory module 108 is not yet address-aligned, so more memory access commands can be sent by memory controller 102 via command / address bus 106 to complete the operation. Figure 2 The example sends fewer commands that operate on address-aligned data, performing the same computation.

[0077] Memory access command 316 indicates Figure 3 Other memory access commands in this series of commands. Similar to other memory access commands in this series, and with... Figure 3 In contrast to the broadcast memory access command 212, memory access command 316 addresses a specific memory location "0" within a memory module "0". Therefore, memory address 318 indicates that... Figure 3 Other memory addresses specified by memory access commands in the series. Specifically, similar to other memory addresses specified by other commands in the series, and opposite to broadcast memory address 214, memory address 318 addresses a specific memory location "2" of a memory module "1".

[0078] Therefore, while the PIM solution may not meet the bandwidth limit of the command / address bus 106 for regular computing workloads with a large number of address-aligned memory accesses, it can meet such limitations for sparse or irregular computing workloads with many non-address-aligned memory accesses, because the memory controller 106 can send a greater number of memory access commands via the command / address bus 106 to accomplish the same computation.

[0079] The PIM solution is supported by near memory address generation, which allows addresses to be generated close to memory, enabling parallel memory access across multiple memory modules 108 using broadcast or multicast memory access commands even when data access is not yet address aligned.

[0080] Memory module

[0081] Figure 4 This is a schematic diagram of an exemplary memory module without near-memory address generation capability. Memory module 408 may include a memory structure 420 providing multiple memory locations, where memory location 410 is an example. Memory module 408 may be electrically connected to memory controller 402 via a command / address bus. For example, memory module 408 may be connected to memory controller 102 via command / address bus 106. Figure 1 One of the memory modules 108 in the memory controller. Therefore, the memory controller 402 can be, for example... Figure 1 The memory controller 102 in the middle.

[0082] In this example, memory structure 420 has a grid or table configuration, where memory locations are arranged in rows and columns. However, memory structure 420 can be implemented as an array of memory locations. Alternatively, memory structure 420 can be a three-dimensional structure, such as a grid or table-style memory structure stacked on top of each other. Memory structure 420 is not limited to any particular arrangement of memory locations, and virtually any memory structure with addressable memory locations can be used.

[0083] The memory module 408 also includes a PIM execution unit 422 for performing near-memory computations on data accessed from memory cells of the memory structure 420. For example, the PIM execution unit 422 may encompass one or more multiply-accumulate units, one or more reducers for matrix-vector multiplication operations, or other subunits for operating on data accessed from the memory structure 420.

[0084] In operation, the memory controller 402 can send memory access commands via the command / address bus 106 for reception at the memory module. The memory access command can specify an address. The specified address can identify a memory location (e.g., 410) in the memory structure 420 of the memory module 408. For example, the specified address can have a row address component in the input row decoder 424 and a column address component in the input column address decoder 426. The combination of the row address and column address specifies the memory location to be accessed (e.g., 410). During a load (or read) operation, data at the memory location is sent to the PIM execution unit 422 and / or sent back to the host (e.g., the memory controller 402) for calculations based thereon. Each memory location of the memory structure 420 can store one or more bytes of data.

[0085] Memory module 408 may include, for simplicity, items not included in... Figure 4 Other components shown include line buffers and input / output drivers.

[0086] Memory module with near address generation unit

[0087] Figure 5 This is a schematic diagram of an exemplary memory module 508 including a near memory address generation unit 528, which generates a near memory address for a memory access command specifying incomplete address information. Figure 4 Similar to memory module 408, memory module 508 may include a memory structure 520 providing multiple memory locations, where memory location 510 is an example. Memory module 508 may be electrically connected to memory controller 502 via a command / address bus. For example, memory module 508 may be connected to memory controller 102 via command / address bus 106. Figure 1 One of the memory modules 108 in the memory controller 502. Therefore, the memory controller 502 can be, for example... Figure 1 The memory controller 102 is included. The memory module 508 may include a PIM execution unit 522, a row decoder 524, and a column decoder 526, and is similar to... Figure 4 The memory module 408 in it.

[0088] However, with Figure 4 Unlike memory module 408, memory module 508 may include a near memory address generation unit 528 for generating near memory addresses for memory access commands specifying incomplete address information. Within the near memory address generation unit 528 are a row address register file (or simply "RARF") 530 and a column address register file (or simply "CARF") 532. RARF 530 may store one or more row addresses for input into row decoder 524 when selected for generation by row selector 534. Similarly, CARF 532 may store one or more column addresses for input into column decoder 526 when selected for generation by column selector 536. Optionally, it may be advantageous to incorporate the near memory address generation unit into an integrated circuit separate from (but still adjacent to) memory module 508, as this would allow for the use of a conventional memory module, resulting in potentially beneficial costs.

[0089] Row selector 534 and column selector 536 can be controlled in response to memory access commands sent by memory controller 502 via the command / address bus. Specifically, row selector 534 and column selector 536 allow memory controller 502 to send memory access commands with incomplete address information, for which row address and / or column address are generated by RARF 530 and / or CARF 532, respectively. Row selector 534 and column selector 536 also allow memory controller 502 to send memory access commands for which address information is not generated by the proximity memory address generation unit 582 to the memory.

[0090] For example, memory controller 502 may send a memory access command specifying an address, which includes both row and column addresses. In this case, row selector 534 may be configured to accept the row address specified in the memory access command as input and not accept the row address generated by RARF 530 for the command as input. Similarly, column decoder 526 may be configured to accept the column address specified in the memory access command as input and not accept the column address generated by CARF 532 for the command as input. Alternatively, in the context of this example, as in the case of DRAM, phase-change memory, resistive memory, or other memory structures that organize memory locations into rows and columns, memory controller 502 may send a memory access command specifying a row address, followed by another memory access command specifying a column address.

[0091] As another example, memory controller 502 may send a memory access command with incomplete address information, which does not specify a row address or a column address. In this case, row selector 534 may be configured to accept the row address generated by RARF 530 for the command as input, and column selector 536 may be configured to accept the column address generated by CARF 532 for the command as input.

[0092] As another example, memory controller 502 may send a first memory access command with incomplete address information that does not specify a row address, followed by a second memory access command with incomplete address information that does not specify a column address. In this case, row selector 534 may be configured to accept the row address generated by RARF 530 for the first command as input, and column selector 536 may be configured to accept the column address generated by CARF 532 for the second command as input.

[0093] As another example, memory controller 502 may send a first memory access command with incomplete address information that does not specify a row address, followed by a second memory access command specifying a column address. In this case, row selector 534 may be configured to accept the row address generated by RARF 530 for the first command as input, and column selector 536 may be configured to accept the column address specified in the second command as input, and not accept the column address generated by CARF 532 for the second command as input.

[0094] As another example, memory controller 502 may send a first memory access command specifying a row address, followed by a second memory access command with incomplete address information and no specified column address. In this case, row selector 534 may be configured to accept the row address specified in the first command as input and not accept the row address generated by RARF 530 for the first command as input, and column selector 536 may be configured to accept the column address generated by CARF 532 for the second command as input.

[0095] As another example, the memory controller 502 may send a memory access command with incomplete address information, specifying either a row address or a column address but not the other. In this case, one of the row selector 534 or the column selector 536 may be configured to accept the row address or column address generated by RARF 530 or CARF 532 for the command as input, and the other of the row selector 534 or the column selector 536 may be configured to accept the row address or column address specified in the memory access command as input.

[0096] Program address command

[0097] The row address and / or column address are pre-programmed in RARF 530 and / or CARF 532 by one or more program address commands sent by memory controller 502 via the command / address bus. Then, for one or more subsequent memory access commands with incomplete address information sent by memory controller 502, the pre-programmed address can be generated near the memory by near memory address generation unit 528.

[0098] For example, via the command / address bus, the memory controller 502 can send a program address command specifying an address to the memory module 508, which includes both row and column addresses. The row address can be stored in the RARF 530, and the column address can be stored in the CARF 532. Then, one or more subsequent memory access commands with incomplete address information can be sent by the memory controller 502 to the memory module 508 via the command / address bus. For each of these subsequent memory access commands with incomplete address information, a pre-programmed row address and a pre-programmed column address can be generated by the RARF 530 and CARF 532 close to the memory.

[0099] As another example, via the command / address bus, the memory controller 502 can send a program address command to the memory module 508 specifying either a row address or a column address, but not both. If a row address is specified, it can be stored in the RARF 530. Conversely, if a column address is specified, it can be stored in the CARF 532. Then, one or more subsequent memory access commands with incomplete address information can be sent by the memory controller 502 to the memory module 508 via the command / address bus. For each of these subsequent memory access commands with incomplete address information, a pre-programmed row address or a pre-programmed column address can be generated close to the memory by the RARF 530 or CARF 532, respectively. Subsequent memory access commands can specify the other of the pre-programmed row address or column address generated close to the memory.

[0100] The RARF 530 and / or CARF 532 contain multiple memory locations capable of storing multiple pre-programmed addresses. Each such memory location can be associated with an index (e.g., 0, 1, 2, etc.). The pre-programmed address can be referenced by its index in memory access commands with incomplete address information.

[0101] For example, a program address command can be sent by the memory controller 502 to preprogram the row address at index 1 of the RARF 530 and the column address at index 1 of the CARF 532. Subsequent memory access commands sent by the memory controller 502 can then specify index 1, and for that command, the preprogrammed row and column addresses at index 1 stored in the RARF 530 and CARF 532, respectively, can be generated close to the memory.

[0102] Memory access commands can specify different indices. For example, a memory access command can specify an index for RARF 530 and a different index for CARF 532. In this case, a pre-programmed row address can be generated close to the memory at the specified index for RARF 530, and a pre-programmed column address can be generated close to the memory at a different index for CARF 532.

[0103] Memory access commands can specify only an index for RARF 530 or only an index for CARF 532. In this case, a preprogrammed row address or a preprogrammed column address can be generated near the RARF 530 or CARF 532 index specified by the command, and the command can also specify the other of the row address and column address.

[0104] As indicated, once the RARF 530 and / or CARF 532 have been pre-programmed or otherwise stored with an address, subsequent memory access commands with incomplete address information can be sent by the memory controller 502 via the command / address bus without specifying the address stored in each such subsequent memory access command. As a result, subsequent memory access commands do not consume the command / address bus bandwidth they would have consumed had the stored address been specified in the memory access command, because fewer bits can be transmitted with each such command.

[0105] Different memory modules (e.g., 108-0, 108-1, 108-2) are pre-programmed or otherwise store these different row / column addresses with different row and / or column addresses in each of their respective RARFs and / or CARFs. This allows a single subsequent broadcast memory access command with incomplete address information to be sent by the memory controller 502 via the command / address bus to access data at each of the different memory modules (e.g., stored at different row and / or column addresses) where the data is not yet address-aligned, by generating row and / or column addresses using the nearby memory stored in the respective RARFs and / or CARFs.

[0106] The pre-programming of row and / or column addresses in the RARF 530 and CARF 532 by the program address commands sent by the memory controller 502 is performed during idle cycles of the command / address bus. Then, for multiple subsequent memory access commands sent by the memory controller 502, the pre-programmed row and / or column addresses are repeatedly generated near the memory by the RARF 530 and / or CARF 532. In other words, due to the near-memory address generation capability provided by the RARF 530 and / or CARF 532, the command / address bus bandwidth consumed by sending program address commands via the command / address bus can be distributed among multiple subsequent memory access commands.

[0107] Address Calculation Unit

[0108] The near memory address generation unit includes an address calculation unit, which provides near memory address calculation functions to the memory module. Figure 6 This is a schematic diagram of an exemplary memory module 608 including a near memory address generation unit 628, which generates a near memory address for a memory access command specifying incomplete address information. Figure 5 Similar to memory module 508, memory module 608 may include a memory structure 620 providing multiple memory locations, where memory location 610 is an example. Memory module 608 may be electrically connected to memory controller 602 via a command / address bus. For example, memory module 608 may be connected to memory controller 102 via command / address bus 106. Figure 1 One of the memory modules 108 in the memory controller 602. Therefore, the memory controller 602 can be, for example... Figure 1 The memory controller 102 is included. The memory module 608 may include a PIM execution unit 622, a row decoder 624, a column decoder 626, a RARF 630, a CARF 632, a row selector 634, and a column selector 646, and is similar to... Figure 5 The memory module 508 in the middle.

[0109] However, with Figure 5 Unlike memory module 508, memory module 608 includes address calculation unit 638 electrically connected to RARF 630, CARF 632 and PIM unit 622.

[0110] Address calculation unit 638 is configured to increment or decrement the row and / or column addresses stored in RARF 630 and CARF 632, respectively, using a configurable step size. The step size may be pre-configured within address calculation unit 638. Alternatively, the step size may be pre-programmed into address calculation unit 638 via a memory access command with incomplete address information sent from memory controller 602 via the command / address bus, wherein the step size is specified by the command.

[0111] For example, for a memory access command with incomplete address information received at memory module 608, after accessing a memory location in memory structure 620, address calculation unit 638 may increment, decrement, or multiply by the column address stored in CARF 632. Similarly, additionally or optionally, for a memory access command, after accessing a memory location in memory structure 520, address calculation unit 638 may increment, decrement, or multiply by the row address stored in RARF 630. The next memory access command with incomplete address information received at memory module 608 may generate an incrementing, decrementing, or multiplying address by RARF 630 and / or CARF 632. This process can continue for multiple memory access commands, thereby avoiding the need to consume command / address bus bandwidth by sending multiple program address commands. Instead, the second and subsequent addresses may be calculated by address calculation unit 628 using a single starting address specified by the initial program address command, and a series of simple calculations performed by address calculation unit 628 using a stride. In addition to incrementing by step size, the address calculation unit 628 can also support many other address calculation methods, including decrementing by step size, multiplying by step size, and so on.

[0112] Address calculation unit 628 can access or be provided with values ​​calculated by or accessed from memory structure 620 by PIM execution unit 622 associated with memory structure 620 for use as addresses. Address calculation unit 628 can then use such addresses to calculate row and column addresses to be programmed into RARF 630 and CARF 632. Examples of such use may include: reading address offset values ​​from memory structure 620; adding the address offset values ​​to a predetermined base address of the data structure using PIM execution unit 622; and using address calculation unit 628 to generate the resulting row and column addresses to be programmed into RARF 630 and CARF 632.

[0113] This can be performed by address calculation unit 628 instead of PIM unit 622: adding the offset read from memory structure 620 to the base address stored in RARF 630 or CARF 632.

[0114] The address calculation unit in multiple memory modules allows a single memory access command with an incomplete address to be sent by the memory controller or host via the command / address bus to apply to a different address in each memory module, since the offset associated with each memory module, as well as the possible base address, can differ because they are stored separately in their respective RARF / CARF and / or corresponding memory structures. The address calculation unit further reduces the bandwidth requirements of the command / address bus because the offset is read from the memory structure associated with each memory module. Therefore, there is no need to transmit the offset via the command / address bus. Note that reading the offset from the memory structure can also be performed via broadcast or multicast memory access commands sent to multiple memory modules, further reducing the command / address bus bandwidth requirements.

[0115] Exemplary command series

[0116] Figure 7 An exemplary series of program address commands 740-1, 740-2, and 740-3 and memory access commands 742-1, 742-2, and 742-3 are shown for accessing data stored at memory locations that are not yet address-aligned among memory modules 108. In this example, there are three memory access commands 742-1, 742-2, and 742-3 for accessing different memory locations within memory module 108 that are not yet address-aligned. The memory module 108 pre-programs the starting addresses for near-memory address generation using the program address commands 740-1, 740-2, and 740-3 to be used by the first memory access command 742-1. Then, for the second memory access command 742-2, the starting addresses are incremented by a fixed step (e.g., 1). Then, for the third memory access command 742-3, these addresses are again incremented by a fixed step. Thus, utilizing the near-memory address generation capability of the memory module 108, calculations can be performed using only six commands sent by the memory controller 102 via the command / address bus 106, which is consistent with the above. Figure 3 The nine commands required in the middle are the opposite.

[0117] In detail, the program address command 740-1 is sent by the memory controller 102 to the memory module 108-0 via the command / address bus 106 leading to the memory module 108-0 to set the module relative start address of the module 108-0 to "0".

[0118] Then, the program address command 740-2 is sent by the memory controller 102 to the memory module 108-1 via the command / address bus 106 leading to the memory module 108-1 to set the module relative start address of the module 108-1 to "1".

[0119] Then, the program address command 740-2 is sent by the memory controller 102 to the memory module 108-2 via the command / address bus 106 leading to the memory module 108-2 to set the module relative start address of the module 108-2 to "2".

[0120] Then, via command / address bus 106, memory controller 102 broadcasts memory access command 742-1 with incomplete address information (e.g., no address) to each of memory modules 108-0, 108-1, and 108-2. Command 742-1 specifies operation OP-1 and operand X-1. For example, operation OP-1 could be a PIM command, such as multiplication or addition. While operation OP-1 could be a PIM command, it could also be a simple memory load command or other commands. For example, assuming operation OP-1 is multiplication and operand X-1 is the number 5, then based on the module-relative start address pre-programmed by program address command 740, command 742-1 multiplies the data stored at memory location "0:0" in module 108-0 by 5, multiplies the data stored at memory location "1:1" in module 108-1 by 5, and multiplies the data stored at memory location "2:2" in memory module 108-2 by 5. After each multiplication, the module relative starting address of memory module 108 can be incremented by a fixed step (e.g., 1), so that the module relative starting addresses of memory modules 108-0, 108-1 and 108-2 are now “1”, “2” and “3”, respectively.

[0121] Then, via the command / address bus 106, the memory controller 102 broadcasts a memory access command 742-2 with incomplete address information (e.g., no address) to each of the memory modules 108-0, 108-1, and 108-2. Command 742-2 specifies operation OP-1 and a different operand X-2. For example, assuming operation OP-1 is multiplication and operand X-2 is the number 8, then based on the current module's relative starting address, command 742-2 multiplies the data stored at memory location "0:1" in module 108-0 by 8, multiplies the data stored at memory location "1:2" in module 108-1 by 8, and multiplies the data stored at memory location "2:3" in memory module 108-2 by 8. After each multiplication, the module relative starting address of memory module 108 can be incremented by a fixed step (e.g., 1), so that the module relative starting addresses of memory modules 108-0, 108-1 and 108-2 are now “2”, “3” and “0”, respectively.

[0122] Then, via the command / address bus 106, the memory controller 102 broadcasts a memory access command 742-3 with incomplete address information (e.g., no address) to each of the memory modules 108-0, 108-1, and 108-2. Command 742-3 specifies operation OP-1 and a different operand X-3. For example, assuming operation OP-1 is multiplication and operand X-3 is the number 0, then based on the current module's relative starting address, command 742-3 multiplies the data stored at memory location "0:2" in module 108-0 by 0, multiplies the data stored at memory location "1:3" in module 108-1 by 0, and multiplies the data stored at memory location "2:0" in memory module 108-2 by 0.

[0123] Exemplary command format

[0124] As an example, memory controller 102 and memory module 108 may support the following commands to support near memory address generation:

[0125] memory_access_command operation,[mask]

[0126] program_[row,col]_address_commandmemory_module_id,module_relative_address

[0127] The first command described above represents an exemplary broadcast or multicast memory access command format with incomplete address information, used to send to two or more memory modules sharing a command / address bus, such as DRAM libraries within a channel of HBM memory. If it is a broadcast command, the command can be used by all memory modules sharing the command / address bus. If it is a multicast command, the command specifies a bitmask as a parameter to select a subset of two or more memory modules that share the command / address bus pointed to by the command. Optionally, the command may have a mask that suppresses the command at a subset of all memory modules.

[0128] Broadcast or multicast memory access commands can also specify the operation type as a parameter. This operation could be, for example, a memory load. However, if the memory module has PIM capability, PIM operations such as multiplication or addition can be specified for near-memory processing. It is worth noting that, unlike conventional memory access commands, the above memory access commands can be sent without specifying any address information or only specifying incomplete address information. Furthermore, unlike conventional memory access commands, the near-memory address generation unit of the memory module can generate a lost address or a lost address portion near the memory in response to the above memory access commands.

[0129] The second command is an exemplary program address command format for sending to a memory module via a shared command / address bus. This command accepts the identifier of the memory module to which the command is targeted as a parameter. The command also accepts address parameters, which can be row addresses, column addresses, row and column addresses, or other module-relative addresses that may vary depending on the type of program address command. For example, different types of program address commands can be used to program RARF and CARF separately using row addresses and column addresses, respectively.

[0130] Although the program address commands presented herein program each memory module individually, alternatively, broadcast or multicast program address commands with associated masks can program multiple memory modules having the same module-relative addresses. Furthermore, as mentioned, other variations of the program address commands may include programming row and column addresses or other module-relative addresses separately.

[0131] Other considerations

[0132] Certain computer memory technologies, such as Hybrid Memory Cube (or simply "HMC") and HBM memory, can stack multiple memory layers on top of a basic die. Such a basic die can potentially possess near-memory computing capabilities, such as... Figure 5 and Figure 6 The near memory address generation units 528 and 628.

[0133] The near memory address generation units 528 and 628 can be placed close to the memory array / group or conventional DRAM in the memory layer of a three-dimensional (3-D) stack memory.

[0134] The implementation schemes can also be applied to other forms of memory besides DRAM, such as non-volatile memory, flash memory and embedded DRAM.

[0135] While some implementations describe the state and logic used to associate a single starting address to be used with any of the commands in the proposed memory access commands, copying the proposed state and logic for multiple PIM operations can allow multiple starting addresses to be tracked by the nearest memory address generation unit or different PIM operations, such as tracking different addresses in each memory module for multiplication and addition operations.

[0136] The memory controller 102 can work with the memory modules 108 to employ a deterministic double data rate (or simply "DDR"). According to some embodiments, DDR can be supported by appropriately synchronizing the memory structures 108. For example, a memory access command requiring DRAM row open (activation) may take longer than a DRAM row cache hit. To allow broadcast or multicast memory access commands with incomplete address information to access different addresses across different memory modules, the memory controller 102 can be augmented to synchronize row open between different memory modules.

[0137] in conclusion

[0138] The term "implementation" as used herein means that a specific feature, structure, or characteristic is described in connection with at least one embodiment of the invention and may be included in at least one embodiment of the invention. The phrases "in an embodiment" and "according to an embodiment" appearing in different places in the specification do not necessarily refer to the same embodiment, nor are they separate or alternative embodiments that are mutually exclusive with other embodiments.

[0139] Although some of the various figures show multiple logical levels in a specific order, levels that are not dependent on order can be reordered, and other levels can be combined or decomposed. While some reorderings or other groupings are specifically mentioned, the orderings and groupings presented in this paper are not an exhaustive list of alternatives.

[0140] In the foregoing specific embodiments and the appended claims, although in some instances the terms first, second, etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first computing device may be referred to as a second computing device, and similarly, a second computing device may be referred to as a first computing device. Both the first computing device and the second computing device are computing devices, but they are not the same computing device.

[0141] As used in the foregoing detailed description and appended claims, the singular forms “a” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise. As used in the foregoing detailed description and appended claims, the term “and / or” refers to and covers any and all possible combinations of one or more of the associated listed items.

[0142] As used in the foregoing detailed embodiments and the appended claims, the terms “based on,” “according to,” “comprising,” “including,” and / or “containing” designate the presence of a specified feature, integer, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0143] As used in the foregoing detailed embodiments and the appended claims, the term "if" is optionally interpreted as meaning "when," "after," "in response to determination," "in response to detection," or "according to determination," depending on the context. Similarly, the phrase "if determined" or "if [prescribed condition or event] is detected" is optionally interpreted as meaning "after determining," "in response to determination," "after detecting [prescribed condition or event]," "in response to detecting [prescribed condition or event]," or "according to the determination of detecting [prescribed condition or event]," depending on the context.

[0144] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details, which may vary depending on the specific implementation. Therefore, this specification and the accompanying drawings should be regarded as illustrative rather than restrictive.

Claims

1. A memory controller, comprising: Command logic configured to send memory access commands with or without address information via a connection between the memory controller and two or more memory modules, the memory access commands being used to access data stored at two or more memory locations in the two or more memory modules, the two or more memory locations corresponding to two or more near memory generated addresses generated based on the memory access commands with or without address information, wherein each of the two or more memory locations is located at a different memory module in the two or more memory modules.

2. The memory controller of claim 1, wherein at least two of the two or more near memory generated addresses are different addresses.

3. The memory controller according to claim 1, wherein: The command logic is configured to: (a) send a first program address command specifying a first address to the first memory module of the two or more memory modules via the connection; (b) sending a second program address command with a second address to a second memory module of the two or more memory modules via the connection; and The command logic is configured to send the memory access command to access data stored at the two or more memory locations based on the first address and the second address.

4. The memory controller of claim 1, wherein the memory access command specifies the specific operation to be performed by two or more in-memory processor units in the two or more memory modules on the data stored at the two or more memory locations.

5. The memory controller according to claim 1, wherein the partial address information includes: (a) The row address shared by the two or more memory locations; Or (b) the column address shared by the two or more memory locations.

6. The memory controller of claim 1, wherein the memory access command includes a mask that suppresses the memory access command at one or more of a plurality of memory modules sharing the connection.

7. The memory controller according to claim 1, wherein: The command logic is configured to send a program address command specifying a specific address to the two or more memory modules via the connection; and The command logic is configured to send the memory access command via the connection for accessing data stored at the two or more memory locations based on the specific address.

8. The memory controller of claim 1, wherein the command logic is configured to send a plurality of memory access commands via the connection for accessing data stored at two or more memory locations in two or more memory modules, the two or more memory locations corresponding to two or more near memory generation addresses.

9. The memory controller of claim 1, wherein the command logic is configured to send the memory access command via the connection, the memory access command specifying a register file index for accessing data stored at the two or more memory locations; and wherein the register file index is usable at the two or more memory modules to generate the two or more near memory generation addresses.

10. A memory module, comprising: A memory structure having multiple addressable memory locations; and A near memory address generation unit is configured to generate an address corresponding to a memory location among a plurality of memory locations based on a memory access command with incomplete address information received via a connection at the memory module. The incomplete address information is used to access data stored at a set of two or more memory locations in a set of two or more memory modules, and the set of two or more memory locations corresponds to a set of two or more near memory generated addresses. The group of two or more memory modules includes the memory module; The set of two or more memory locations includes the memory locations; and The set of two or more near memory generated addresses includes the address.

11. The memory module of claim 10, wherein the near memory address generation unit includes a register file configured to store the address corresponding to the memory location based on a program address command specifying the address corresponding to the memory location received at the memory module via the connection.

12. The memory module according to claim 10, wherein: The address is the first address; The memory location is the first memory location; The near-memory address generation unit includes a register file, which is capable of storing a second address corresponding to a second memory location among the plurality of memory locations; and The near-memory address generation unit includes an address calculation unit, which is capable of performing at least one of the following: The second address is incremented by a step size to calculate the first address. The first address is calculated by decreasing the second address by a step size, or The second address is multiplied by the step size to calculate the first address.

13. The memory module of claim 10, further comprising an in-memory processor execution unit; and wherein the near memory address generation unit comprises an address calculation unit configured to generate the address corresponding to the memory location based on a value calculated by the in-memory processor execution unit.

Images