Dual-mode computing memory controller and memory system for PIM-DRAM

Abstract

The invention belongs to the technical field of semiconductors, and particularly relates to a dual-mode computing memory controller and a memory system for a PIM-DRAM. The dual-mode computing memory controller used for the PIM-DRAM comprises an address conversion module used for processing address mapping between a host and a PIM core; the command translation module is used for converting the host command into a PIM operation instruction; the command generation module is used for generating an address mapping command; the data translation module is used for managing data streams among the DDR bus, the on-chip bus and the DRAM; the multi-mode state machine is used for dynamically switching to a memory mode or a PIM mode according to different memory commands sent by the host; and the dual-mode computing access module is used for selecting an access space of the DRAM according to the address mapping command or the data route. According to the method, efficient and low-delay PIM operation is achieved by dynamically switching the memory and the calculation mode, meanwhile, the method is compatible with a traditional DDR protocol and can be directly integrated to an existing system, and a host interface does not need to be modified.

Description

Technical Field

[0001] The present invention belongs to the field of semiconductor technology, and particularly relates to a dual-mode computing memory controller and memory system for PIM-DRAM. Background Art

[0002] PIM-DRAM is a technology that integrates computing units into DRAM memory, aiming to reduce the latency of data transfer and improve computing efficiency. This technology reduces the number of data transfers between storage and the processor, thereby reducing latency and improving efficiency.

[0003] The memory controller of the existing PIM-DRAM architecture cannot efficiently switch between the traditional storage mode (also known as the Memory Mode) and the computing mode (also known as the PIM Mode), nor can it support the parallel operation of host access and computing units simultaneously, resulting in poor compatibility or low computing efficiency and limiting flexibility. At the same time, the memory controller of the existing PIM-DRAM architecture lacks support for multi-bank synchronous operations, limiting the large-scale parallel computing ability. And in the PIM mode, the memory state needs to be frequently switched, resulting in increased latency. Summary of the Invention

[0004] In view of the technical problem that the memory controller of the existing PIM-DRAM architecture cannot efficiently switch between the traditional storage mode and the computing mode, the purpose of the present invention is to provide a dual-mode computing memory controller and memory system for PIM-DRAM.

[0005] To solve the foregoing technical problems, a first aspect of the present invention provides a dual-mode computing memory controller for PIM-DRAM, and the dual-mode computing memory controller for PIM-DRAM includes:

[0006] An address conversion module for processing the address mapping between the host and the PIM core;

[0007] A command translation module for converting host commands into PIM operation instructions;

[0008] A command generation module, respectively connected to the address conversion module and the command translation module, for generating address mapping commands;

[0009] A data translation module for managing the data flow between the DDR bus, the on-chip bus and the DRAM;

[0010] A multi-mode state machine, respectively connected to the address conversion module, the command translation module, the command generation module and the data translation module, for dynamically switching the dual-mode computing memory controller to the memory mode or the PIM mode according to different memory commands sent by the host;

[0011] A dual-mode computing access module, which is respectively connected to the command generation module and the data translation module, and is used to select the access space of DRAM according to the address mapping command or data routing.

[0012] Optionally, in the dual-mode computing memory controller for PIM-DRAM as described above, when the multi-mode state machine switches the dual-mode computing memory controller to the memory mode, the dual-mode computing memory controller allows the host to access and write data to a single DRAM bank in the DRAM main memory space through standard DDR commands.

[0013] Optionally, in the dual-mode computing memory controller for PIM-DRAM as described above, the PIM mode includes a full-bank mode and a full-bank PIM mode;

[0014] When the multi-mode state machine switches the dual-mode computing memory controller to the full-bank mode, the dual-mode computing memory controller allows the host to configure and program the parameter registers of the PIM core;

[0015] When the multi-mode state machine switches the dual-mode computing memory controller to the full-bank PIM mode, the dual-mode computing memory controller allows the PIM core to access and write data to the computing memory space in DRAM.

[0016] Optionally, in the dual-mode computing memory controller for PIM-DRAM as described above, when the multi-mode state machine switches the dual-mode computing memory controller to the full-bank mode, the dual-mode computing memory controller allows the host to write static weight data and parallelly load the same PIM core program into all processing units, thereby improving the configuration efficiency.

[0017] Optionally, in the dual-mode computing memory controller for PIM-DRAM as described above, when the multi-mode state machine switches the dual-mode computing memory controller to the full-bank mode, the dual-mode computing memory controller allows the host to simultaneously access all DRAM banks in a memory channel through a single memory transaction for parallel configuration of the PIM core. The dual-mode computing memory controller broadcasts the memory commands, row addresses, and column addresses issued by the host to all DRAM banks in a memory channel, thereby achieving synchronous operation.

[0018] Optionally, in the dual-mode computing memory controller for PIM-DRAM as described above, when the multi-mode state machine switches the dual-mode computing memory controller to the full-bank PIM mode, the dual-mode computing memory controller allows the host to simultaneously access all DRAM banks in a memory channel through a single memory transaction and parallelly execute the programmed PIM cores.

[0019] To solve the foregoing technical problems, a second aspect of the present invention provides a memory system, which includes at least one intelligent memory processing unit. The intelligent memory processing unit includes a 3D-stacked DRAM storage layer and a logic computing layer. The memory in the intelligent memory processing unit is divided into one or more independent memory channels. Each memory channel includes a number of DRAM storage units of the DRAM storage layer. A single DRAM storage unit includes a number of DRAM banks, and each of the DRAM banks is correspondingly connected to a memory controller;

[0020] The memory controller adopts the dual-mode computing memory controller provided in the first aspect of the present invention.

[0021] The positive and progressive effects of the present invention are as follows:

[0022] 1. By dynamically switching between the memory and computing modes, the present invention realizes efficient and low-latency PIM operations, and at the same time is compatible with the traditional DDR protocol, and can be directly integrated into the existing system without modifying the host interface.

[0023] 2. In the full-bank PIM mode, all DRAM banks of the present invention can synchronously execute the programmed PIM cores, greatly improving the computing throughput, and can be applied to large-scale parallel computing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Referring to the accompanying drawings, the disclosure of the present invention will become more apparent. It should be understood that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In the figures:

[0025] Figure 1 is a schematic structural diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The following uses specific specific examples to illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0027] It should be noted that, without conflict, the following embodiments and the features in the embodiments may be combined with each other.

[0028] In the description of the present invention, it should be noted that for orientation terms, such as the terms "outer side", "middle section", "inner", "outer", etc., which indicate the orientation and positional relationship are based on the orientation or positional relationship shown in the drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and should not be construed as limiting the specific protection scope of the present invention.

[0029] In addition, such terms as "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, the meanings of "several" and "a number of" are two or more, unless otherwise specifically defined.

[0030] Referring to Figure 1 , an embodiment of the present invention provides a dual-mode compute memory controller for PIM-DRAM. The dual-mode compute memory controller (Dual-Mode Compute Memory Controller, DMCMC) includes an address translation module (Address Translation Module, ATM), a command translation module (Command Translation Module, CTM), a command generation module (Command Generation Module, CGM), a data translation module (DataTranslation Module, DTM), a multi-mode state machine (Multi-Mode State Machine, MMSTM), and a dual-mode compute access module (Dual-Mode Compute Access Module, DCAM, also referred to as DCA Switch).

[0031] The address translation module is used to process the address mapping between the host (HOST) and the PIM core. The command translation module is used to convert the host command into a PIM operation instruction. The cooperation operation between the host and the PIM core is supported by the address translation module and the command translation module.

[0032] The command generation module is respectively connected to the address translation module and the command translation module. The command generation module is used to generate an address mapping command. The cooperation of the address translation module, the command translation module, and the command generation module is responsible for address mapping for the three paths corresponding to the memory mode and the PIM mode.

[0033] The data translation module is used to manage the data flow between the DDR bus, the on-chip bus, and the DRAM. That is, the data translation module is responsible for managing the direct data transfer between the DDR, the internal data bus, and the DRAM.

[0034] The multi-mode state machine is respectively connected to the address translation module, the command translation module, the command generation module, and the data translation module. The multi-mode state machine is used to dynamically switch the dual-mode computing memory controller to the memory mode or the PIM mode according to different memory commands sent by the host. Among them, the PIM mode is the computing mode. By switching the state machine according to different operating modes by the multi-mode state machine, corresponding control flows are generated, and finally seamless switching between the memory mode and the PIM mode is achieved.

[0035] The dual-mode computing access module is respectively connected to the command generation module and the data translation module. The dual-mode computing access module is used to select the access space of the DRAM according to the address mapping command or data routing. In the PIM-DRAM architecture, the DRAM usually includes the DRAM main memory space (DRAM MAIN SPACE), the DRAM computing memory space (DRAM PIM SPACE), and the parameter register (SMPU Register). The dual-mode computing access module is used to select which area of the DRAM the data and commands reach.

[0036] Through the collaborative operation of the above-mentioned address translation module, command translation module, command generation module, data translation module, multi-mode state machine, and dual-mode computing access module, the present invention realizes data routing and address mapping of the dual-mode computing memory controller between the memory mode and the PIM mode. In particular, the multi-mode state machine realizes millisecond-level mode switching, avoiding the reconfiguration overhead of the traditional architecture. By dynamically switching between the memory and computing modes, efficient and low-latency PIM operations are achieved, while being compatible with the traditional DDR protocol and can be directly integrated into the existing system without modifying the host interface.

[0037] In some embodiments, when the multi-mode state machine switches the dual-mode computing memory controller to the memory mode, the dual-mode computing memory controller allows the host to access through the standard DDR command and write data into a single DRAM bank in the DRAM main memory space.

[0038] The memory mode (Memory Mode) is also called the single-bank (Single-Bank, SB) mode or the single bank mode. The behavior of this mode is similar to that of the traditional DRAM and is a conventional access mode. The host accesses a single DRAM bank through the standard DDR command.

[0039] Specifically, the host chip accesses each memory bank using standard memory commands and addresses, and each memory request targets a specific memory bank. This mode is mainly used for traditional memory read and write operations and is suitable for most conventional application scenarios. In the memory mode, the behavior of the SMPU is exactly the same as that of traditional DRAM, ensuring compatibility with existing systems.

[0040] In some embodiments, the PIM mode includes an all-bank mode and an all-bank PIM mode.

[0041] The PIM mode is also called the compute mode or the all-bank mode. It includes the all-bank mode (AB mode) and the all-bank PIM mode (AB-PIM mode), and supports the synchronous execution of compute instructions for all DRAM banks.

[0042] When the multi-mode state machine switches the dual-mode compute memory controller to the all-bank mode, the dual-mode compute memory controller allows the host to configure and program the parameter registers of the PIM core. The AB mode is mainly used for configuring and programming the PIM core. Preferably, in this mode, the host is allowed to write static weight data and load the same PIM core program into all processing units in parallel, thereby improving the configuration efficiency. The AB mode is a special mode that allows the host to access all DRAM banks in a memory channel simultaneously through a single memory transaction for parallel configuration of the PIM core. In this mode, the memory commands, row addresses, and column addresses issued by the host are broadcast to all DRAM banks in a memory channel, thus achieving synchronous operations.

[0043] When the multi-mode state machine switches the dual-mode compute memory controller to the all-bank PIM mode, the dual-mode compute memory controller allows the PIM core to access and write data to the compute memory space in the DRAM. The AB-PIM mode is the core mode of the SMPU and is used to execute actual PIM (in-memory computing) core tasks. In the AB-PIM mode, each memory transaction not only accesses all DRAM banks in a memory channel but also executes the programmed PIM core in parallel. This means that in a single memory transaction, the processing units in all DRAM banks will execute the same compute task simultaneously, significantly improving the compute efficiency and parallelism.

[0044] In some embodiments, the dual-mode compute memory controller is responsible for data routing and address mapping among three different paths:

[0045] Host access to the DRAM main memory space: This path allows the host to access and write to the main memory space in the DRAM.

[0046] Host configures PIM core parameter registers: This path allows the host to configure the parameter registers of the PIM (in-memory computing) core.

[0047] PIM core accesses DRAM computing memory space: This path allows the PIM core to access and write to the computing memory space in DRAM.

[0048] An embodiment of the present invention also provides a memory system, which includes at least one intelligent memory processing unit (SMPU). The intelligent memory processing unit includes a 3D stacked DRAM storage layer and a logic computing layer. The memory within the intelligent memory processing unit is divided into one or more independent memory channels. Each memory channel includes several DRAM storage units of the DRAM storage layer. A single DRAM storage unit includes several DRAM banks, and each DRAM bank is correspondingly connected to a memory controller. The memory controller adopts the dual-mode computing memory controller provided in the above embodiments of the present invention.

[0049] In the memory system of the present invention, the DMCMC is integrated into the controller of each DRAM bank. The DMCMC includes an ATM module, a CTM module, a CGM module, a DTM module, an MMSTM module, and a DCAM module. The SMPU of the present invention mainly operates in two modes: the memory mode and the PIM mode. The DMCMC manages the mode switching process through the MMSTM module, such as SB→AB→AB-PIM.

[0050] In some embodiments, a mode switching process of the SMPU is as follows:

[0051] The mode switching process of the SMPU starts from the initial state of the SB mode. In this state, the SMPU behaves like ordinary DRAM and processes memory requests from the host.

[0052] When it is necessary to enter the AB mode, the host chip sends a first sequence of memory commands through the dual-mode computing memory controller to switch the SMPU from the SB mode to the AB mode, thereby allowing the host to program the PIM core for all processing units in parallel.

[0053] After inserting the PIM core instructions, the host sends a second sequence of memory commands to switch the SMPU from the AB mode to the AB-PIM mode. In this mode, each memory transaction will execute the programmed PIM core in parallel.

[0054] Once the core execution is completed, the host will send a third sequence of memory commands to switch the SMPU from the AB-PIM mode back to the SB mode, restore the normal memory access mode, and allow the system to continue processing regular memory requests.

[0055] The present invention has been described in detail with reference to the embodiments accompanied by drawings. Those of ordinary skill in the art can make various variations to the present invention based on the above description. Therefore, certain details in the embodiments should not constitute a limitation to the present invention, and the present invention will take the scope defined by the appended claims as the protection scope.

Claims

1. A dual-mode computing memory controller for PIM-DRAM, characterized in that, The dual-mode computing memory controller for PIM-DRAM includes: An address translation module for handling the address mapping between the host and the PIM core; A command translation module for converting host commands into PIM operation instructions; A command generation module, connected to the address translation module and the command translation module respectively, for generating address mapping commands; A data translation module for managing the data flow between the DDR bus, the on-chip bus, and the DRAM; A multi-mode state machine, connected to the address translation module, the command translation module, the command generation module, and the data translation module respectively, for dynamically switching the dual-mode computing memory controller to the memory mode or the PIM mode according to different memory commands sent by the host; A dual-mode computing access module, connected to the command generation module and the data translation module respectively, for selecting the access space of the DRAM according to the address mapping command or data routing; 2. The dual-mode computing memory controller for PIM-DRAM according to claim 1, characterized in that, When the multi-mode state machine switches the dual-mode computing memory controller to the memory mode, the dual-mode computing memory controller allows the host to access and write data to a single DRAM bank in the DRAM main memory space through standard DDR commands.

3. The dual-mode computing memory controller for PIM-DRAM according to claim 1 or 2, characterized in that, The PIM mode includes the all-bank mode and the all-bank PIM mode; When the multi-mode state machine switches the dual-mode computing memory controller to the all-bank mode, the dual-mode computing memory controller allows the host to configure and program the parameter registers of the PIM core; When the multi-mode state machine switches the dual-mode computing memory controller to the all-bank PIM mode, the dual-mode computing memory controller allows the PIM core to access and write data to the computing memory space in the DRAM.

4. The dual-mode computing memory controller for PIM-DRAM according to claim 3, wherein, When the multi-mode state machine switches the dual-mode computing memory controller to the all-bank mode, the dual-mode computing memory controller allows the host to write static weight data and parallelly load the same PIM core program into all processing units, thereby improving the configuration efficiency.

5. The dual-mode computing memory controller for PIM-DRAM according to claim 4, wherein, When the multi-mode state machine switches the dual-mode computing memory controller to the all-bank mode, the dual-mode computing memory controller allows the host to simultaneously access all DRAM banks in a memory channel through a single memory transaction for parallel configuration of the PIM core. The dual-mode computing memory controller broadcasts the memory commands, row addresses, and column addresses issued by the host to all DRAM banks in a memory channel, thereby achieving synchronous operation.

6. The dual-mode computing memory controller for PIM-DRAM according to claim 3, wherein When the multi-mode state machine switches the dual-mode computing memory controller to the all-bank PIM mode, the dual-mode computing memory controller allows the host to simultaneously access all DRAM banks in a memory channel through a single memory transaction and parallelly execute the programmed PIM core.

7. A memory system, characterized in that, The memory system includes at least one intelligent memory processing unit, and the intelligent memory processing unit includes a 3D stacked DRAM storage layer and a logic computing layer. The memory within the intelligent memory processing unit is divided into one or more independent memory channels, and each of the memory channels includes a number of DRAM storage units of the DRAM storage layer. A single DRAM storage unit includes a number of DRAM banks, and each of the DRAM banks is correspondingly connected to a memory controller; The memory controller adopts the dual-mode computing memory controller for PIM-DRAM as described in any one of claims 1 to 6.

Images