Heterogeneous execution method and system of host and near-memory accelerator in near-memory computing architecture

Abstract

The application relates to the technical field of host and near-memory accelerator communication, and discloses a host control and near-memory accelerator heterogeneous execution method and system in a near-memory computing architecture, which comprises three mechanisms of explicit synchronization, unloading-scheduling-returning and implicit synchronization, and comprises the following steps: in the host mode, a Host MC directly sends a DDR command to a memory device, bypasses the command to an NMA MC, and updates a memory state recorded in the NMA MC; in the simultaneous memory access mode and the near-memory mode, the Host MC unloads a memory access request to the NMA MC, the NMA MC schedules the access request unloaded by the host and a local access request of the NMA according to a switching-recovering method, ensures the correctness of the memory state and optimization of row locality, and the NMA MC returns data to the Host MC through a time division multiplexing interrupt and an adaptive batch processing mechanism; and when the simultaneous memory access mode and the near-memory mode end, the NMA MC actively adjusts the memory state according to a recorded memory bank Bank state change instruction. The correctness, consistency and high efficiency of the Host MC and the NMA MC in the simultaneous memory access mode are realized.

Description

Technical Field

[0001] This invention relates to the field of communication technology between host and near-memory accelerator, and in particular to a heterogeneous execution method and system for host and near-memory accelerator in a near-memory computing architecture. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence, big data, and high-performance computing, traditional computing architectures are facing the severe challenge of the memory wall. In traditional storage architectures based on Dual Inline Memory Modules (DIMMs), as shown in the attached diagram... Figure 1 As shown, the host manages multiple independent channels through a memory controller (MC), each channel connecting to several DIMM modules. Each DIMM consists of multiple memory ranks, and each rank is further divided into a bank group (BG) and a bank. The host needs to send activation (ACT), read / write (RD / WR), and precharge (PRE) instructions to access the target data, based on complex timing constraints (such as row activation delay tRCD, precharge time tRP, etc.). To overcome this limitation, Near Memory Processing (NMP) architecture has emerged, as shown in the attached diagram. Figure 2 As shown, this architecture significantly reduces data transfer distance and power consumption by embedding computing units (such as Near Memory Accelerators, NMAs) near the memory, enabling data processing directly in memory, thus improving overall system performance.

[0003] However, the inherent defects of the traditional architecture are reflected in the following aspects: (1) Asynchronous execution conflict. When the host controller and NMA initiate memory access at the same time, the memory controllers (Host MC and NMA MC) lack a synchronization mechanism, which may lead to memory bus conflicts and thus cause DRAM errors; (2) Low row locality utilization. The traditional MC scheduling strategy does not fully optimize the row hit rate of adjacent accesses, resulting in frequent row switching operations, increasing latency and power consumption; (3) Limited scalability. The existing DIMM-NMP architecture usually adopts a single-level (such as Rank level) NMA integration method, which is difficult to support multi-level collaborative computing scenarios.

[0004] Based on this, to alleviate access conflicts between the controller and NMA, the industry has proposed two mainstream solutions, as shown in the attached figure. Figure 3 As shown, however, all of them have significant limitations:

[0005] (1) Shielding Access: This scheme isolates the access paths of the master controller and the NMA through a hardware multiplexer (MUX), ensuring exclusive use of DIMMs by both. For example, the master controller configures the MUX state through a software development kit (SDK) before the NMA executes, and switches back to master controller mode after the NMA completes the calculation. However, its core drawbacks include: storage capacity and bandwidth loss, as some DIMMs need to be reserved for NMA, resulting in a decrease in the overall system storage capacity and available bandwidth; limited flexibility, with stringent requirements for data layout, memory address mapping, and dynamic allocation strategies, making it difficult to adapt to dynamic load scenarios.

[0006] (2) Opportunistic Access: This scheme allows NMA MC to track the master access status in real time and initiate local memory operations "when the opportunity arises." It relies on the NMA workload having a predefined deterministic pattern and avoids master access starvation through probabilistic arbitration. However, its key problems are: narrow applicability, only supporting specific workloads with fixed computation patterns and predictable latency (such as regular matrix operations), and cannot adapt to heterogeneous computing needs; lack of row locality, making it difficult for opportunistic access to take advantage of the locality of consecutive row addresses, and frequent row switching leads to performance degradation; high hardware complexity, requiring the implementation of high-precision state tracking logic in NMA MC, increasing design cost and power consumption.

[0007] Therefore, how to achieve efficient collaboration between the host and the near-memory accelerator (NMA) in a near-memory computing architecture, especially the synchronization and consistency management of memory access, has become a core technical challenge that urgently needs to be solved. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a heterogeneous execution method and system for the main controller and the near-memory accelerator in a near-memory computing architecture. Through explicit synchronization mechanism, unload-schedule-return mechanism and timing optimization strategy, efficient collaboration between the main controller and the near-memory accelerator is achieved.

[0009] On the one hand, a heterogeneous execution method for the main controller and the near-memory accelerator in a near-memory computing architecture is provided, including the following steps:

[0010] S1: In host mode, the host memory controller directly sends DDR commands to the memory device, and at the same time bypasses the commands to the near memory accelerator memory controller, and updates the memory status recorded in the near memory accelerator memory controller in real time.

[0011] S2: In simultaneous memory access mode and near memory mode, the host memory controller offloads memory access requests to the near memory accelerator memory controller through an offload mechanism. The near memory accelerator memory controller schedules the access requests offloaded by the host and the local access requests of the near memory accelerator according to a preset switch-recovery method to ensure the correctness of memory state and row locality optimization. The near memory accelerator memory controller returns data to the host memory controller through time-division multiplexing interrupt and adaptive batch processing mechanism.

[0012] S3: When the simultaneous memory access mode and near memory mode end, the near memory accelerator memory controller actively adjusts the memory state according to the recorded memory bank state change instructions to ensure consistency with the host memory controller state, in order to switch to host mode.

[0013] Further, in step S1, the real-time update of the memory state recorded in the near-memory accelerator memory controller includes:

[0014] The DDR commands are transmitted via the command / address bus. The near memory accelerator memory controller treats the received host commands as commands it issued and processes them accordingly, updating the memory state recorded in the near memory accelerator memory controller so that it can always keep track of the latest memory state.

[0015] Further, in step S2, the host memory controller offloading memory access requests to the near-memory accelerator memory controller includes:

[0016] A new memory command is introduced for timing optimization of the unloading operation. The host memory controller sends an unloading access request to the near memory accelerator memory controller through the command / address bus, and various delays are omitted through timing optimization.

[0017] Preferably, the timing optimization further includes:

[0018] In the unloaded read command RDO, the row activation delay tRCD and timing constraints tRTRs are omitted. At the same time, the physical execution of the row activation instruction ACT is omitted in the unloaded read command RDO, and only its logical state is recorded.

[0019] Insert read operations into the unloaded write command WRO to omit write-to-read latency tWTR, read-to-write latency tRTW, and write latency tWL, in order to maximize bus utilization.

[0020] Further, in step S2, the local memory controller of the near memory accelerator schedules the access requests from the host offloading and the local access requests from the near memory accelerator according to a preset switch-recovery method, including:

[0021] In the near-memory accelerator memory controller, a separate command queue (CMD FIFO) and request queue (REQ FIFO) are set up for each memory bank. Host commands and near-memory accelerator requests are isolated in the command queue (CMD FIFO) and request queue (REQ FIFO), respectively. The queues are switched only when one of the following conditions is met:

[0022] There are no remaining commands or requests in the current queue;

[0023] Issue the PRE command;

[0024] By recording the last BSC command from the command queue CMD FIFO and the request queue REQ FIFO, it is determined whether recovery is needed, and the corresponding BSC command is issued during the switchover recovery to ensure the consistency of the memory state after the switchover.

[0025] Further, in step S2, the time-division multiplexing interrupt includes:

[0026] Interrupt-based signals are used to notify the host memory controller from the near-memory accelerator, and the effective period for each memory column rank to issue an interrupt signal is limited by time-division multiplexing of the interrupt signal to reduce polling latency.

[0027] Preferably, in step S2, the adaptive batch processing mechanism further includes:

[0028] Interrupts are issued in the order in which memory accesses are unloaded, and data is returned in batches when the host memory controller requests the near-memory accelerator memory controller to return data. The batch size is equal to the number of interrupts that unloaded the read.

[0029] Subsequent interrupts must not be triggered if an earlier interrupt has not been completed.

[0030] Multiple read access interrupts are merged into a batch, and the data is returned in batches via a single return instruction (RT) command. The delay of the RT command is dynamically adjusted by the batch size and is executed before other memory access commands.

[0031] On the other hand, a heterogeneous execution system for the host controller and near-memory accelerator in a near-memory computing architecture that implements the above-described method is provided, including a host memory controller and a near-memory accelerator memory controller. The host memory controller includes an unload table, a return unit, and a command encoding module. The near-memory accelerator memory controller includes a bypass path, a configurable FIFO queue, and a switch-recovery unit.

[0032] The unloading table is used to record the number of commands that have been unloaded for each memory bank. When the number of commands reaches a preset threshold, the generation of new commands is paused.

[0033] The return unit is used to manage the timing and priority of the returned data and is configured to dynamically adjust the burst length based on the interrupt count.

[0034] The command encoding module uses reserved fields to encode the return instruction RT command and distinguishes the unload command by reusing the DDR command signal;

[0035] The bypass path is used to directly transmit commands from the host memory controller to the memory device in host mode, while updating the local memory status.

[0036] The configurable FIFO queue is used to dynamically allocate FIFO queue capacity in concurrent mode, store access requests offloaded by the host and local access requests of the near-memory accelerator respectively, and realize queue switching through an arbitrator.

[0037] The switching-recovery unit is used to record BSC commands and generate recovery instructions to ensure the correctness of memory state and mode switching.

[0038] Furthermore, the return unit further includes a physical address mapping module and a priority controller, wherein,

[0039] The physical address mapping module is used to associate interrupt signals with corresponding memory accesses;

[0040] The priority controller is used to set the issuance priority of the return instruction RT command to be higher than that of other DDR commands;

[0041] The capacity allocation of the configurable FIFO queue is achieved in the following way:

[0042] In near-memory mode, the near-memory accelerator memory controller uses the entire request queue REQ FIFO to store near-memory accelerator local access requests. In concurrent mode, 50% of the FIFO queue capacity is used to store host offload commands.

[0043] The alternating scheduling of host offload commands and local access requests is achieved through an arbitrator in each hybrid FIFO unit.

[0044] In addition, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the heterogeneous execution method of the master controller and the near-memory accelerator in the near-memory computing architecture described in any of the above claims.

[0045] Meanwhile, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the heterogeneous execution method of the main controller and the near-memory accelerator in the near-memory computing architecture described in any of the preceding claims.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] This invention uses a bypass path to directly transmit commands from the host memory controller to the memory device in host mode, while updating the local memory state and synchronizing the memory states of the host memory controller and the near-memory accelerator memory controller in real time, thus eliminating asynchronous conflicts.

[0048] This invention optimizes queue scheduling based on the switch-recovery method. In the near-memory accelerator memory controller, an independent command queue (CMD FIFO) and request queue (REQ FIFO) are set up for each memory bank. Host commands and near-memory accelerator requests are isolated in the command queue (CMD FIFO) and request queue (REQ FIFO) respectively, maximizing row locality.

[0049] This invention significantly improves bandwidth utilization by optimizing timing, scheduling more offloading commands, and omitting redundant delay constraints in the offloading commands.

[0050] This invention introduces a new memory command for batch data return in the return mechanism, while using time-division multiplexing for interrupt notification and reusing existing DDR command encoding and bus interface to reduce modification costs. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0052] Figure 1 This is a schematic diagram of a storage architecture based on dual in-line memory (DIMM) modules according to the present invention.

[0053] Figure 2 This is a schematic diagram of a near-memory computing architecture according to the present invention;

[0054] Figure 3 A comparative diagram of existing technical solutions for blocking access and opportunity access.

[0055] Figure 4 This is a flowchart of a heterogeneous execution method between a main controller and a near-memory accelerator in a near-memory computing architecture according to the present invention.

[0056] Figure 5 This is a schematic diagram showing the overall structure of the heterogeneous execution method mechanism between the main controller and the near-memory accelerator in a near-memory computing architecture according to the present invention.

[0057] Figure 6 This is an example diagram of timing optimization in an unloading mechanism of the present invention;

[0058] Figure 7 This is an example diagram of a switch-recovery mechanism in a memory bank scheduling mechanism according to the present invention;

[0059] Figure 8 This is a timing design example diagram for a return mechanism according to the present invention;

[0060] Figure 9 This is a block diagram of the heterogeneous execution structure of the main controller and the near-memory accelerator in a near-memory computing architecture of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] Near Memory Processing (NMP) architecture is a novel computing architecture that integrates Near Memory Accelerators (NMAs) near the memory storage units. In traditional computing architectures, memory and processors are usually separate entities that require data transfer via buses or other means; while NMP can perform data processing in memory, reducing data transfer latency and power consumption, and improving overall computing performance.

[0063] This invention addresses a storage architecture based on Dual Inline Memory Modules (DIMMs). For example... Figure 1The diagram illustrates two channels in a DIMM-based memory architecture. Generally, the host has multiple independent channels, each managed by a memory controller (MC) for memory access. Each channel connects one or more DIMMs via memory buses, which include a command / address (C / A) bus and a data bus. Each DIMM consists of one or two memory ranks, and each rank is further composed of multiple cooperating Dynamic Random Access Memory (DRAM) chips. These chips simultaneously receive commands and addresses, and provide or receive portions of data to form the complete data transmitted on the data bus. DRAM chips can be further divided into bank groups (BGs) and banks. A bank is a two-dimensional memory array containing rows and columns. Because DRAM chips are cooperative, accessing a bank in a particular rank actually accesses the corresponding bank in all DRAM chips within that rank simultaneously.

[0064] When a host wants to access data in a specific row of a specific bank within a specific rank, it needs to send an Activation (ACT) instruction (containing the addresses of BG, Bank, and Row) to that rank to instruct the bank to store the data for that row in the Row Buffer. Then, it sends a Read (RD) instruction or a Write (WR) instruction (containing the address of the Column) to transfer the data. If it's an RD instruction, the data at the corresponding address in the Row Buffer is transferred to the Host MC via the Data Bus; if it's a WR instruction, the Host MC transfers the data to the corresponding address in the Row Buffer and replaces the existing data. When the host no longer needs the data in the Row Buffer or wants to access other rows in that bank, the Host MC needs to send a Precharge (PRE) instruction to store the data from the Row Buffer back to the corresponding row in that bank. Timing constraints between instructions are defined in the specification and must be strictly followed to prevent physical errors.

[0065] The ACT and PRE commands change the Bank state to open and closed, respectively. These are commands that change the Bank state and are called Bank state change (BSC) commands.

[0066] The MC needs to record (1) the Bank status and (2) the memory bus status in real time to (1) calculate the sending time of the next instruction and (2) generate the corresponding DDR commands from the requests. If the information it records is incorrect, memory access errors will occur, such as sending the wrong instruction at the wrong time.

[0067] Furthermore, Near Memory Processing (NMP) architecture is a novel computing architecture that integrates Near Memory Accelerators (NMAs) near the memory storage units. In traditional computing architectures, memory and processors are typically separate entities that require data transfer via buses or other means; while NMP can perform data processing within memory, reducing data transfer latency and power consumption, and improving overall computing performance.

[0068] In existing DIMM-based NMP architectures, NMA is typically integrated into the Rank-level buffer chip, BG-level chip, or Bank-level chip. This invention addresses Rank-level DIMM-NMP, such as... Figure 2 As shown. It is worth mentioning that generally only one level of NMA exists, and this invention only applies to the NMP architecture with only one level of NMA. The NMA architecture generally consists of a compute unit and a local memory controller (Local MC).

[0069] This invention ensures real-time consistency of memory status between the Host MC and the NMA MC, avoids conflicts and errors, supports concurrent access by the host controller and NMA, adapts to diverse computing scenarios, reduces row switching frequency through intelligent scheduling, improves memory bandwidth utilization, is compatible with existing DIMM interface standards, and avoids disruptive modifications to the hardware architecture.

[0070] The specific embodiments of the present invention will be described below with reference to the accompanying drawings and examples.

[0071] First Embodiment

[0072] like Figure 4 and Figure 5As shown, this embodiment provides a heterogeneous execution method for the main controller and the near-memory accelerator in a near-memory computing architecture. It generally includes three mechanisms: explicit synchronization, unloading-scheduling-return, and implicit synchronization. The specific technical solution includes the following steps:

[0073] S1 explicit synchronization mechanism: In Host Mode, the Host Memory Controller (Host MC) directly sends DDR commands to the memory device, while bypassing the commands to the Near Memory Accelerator Memory Controller (NMA MC) to update the memory state recorded in the NMA MC in real time;

[0074] S2 Offload-Schedule-Return Mechanism: In Concurrent Mode and Near-Memory Mode (NMAMode), the Host MC offloads memory access requests to the NMA MC through the offload mechanism. The NMA MC schedules the access requests offloaded by the host and the NMA local access requests according to the preset Switch-Recovery (SR) method to ensure the correctness of the memory state and row locality optimization. The NMA MC returns the data to the Host MC through time-division multiplexing interrupts and adaptive batch processing mechanisms.

[0075] S3 implicit synchronization mechanism: When the simultaneous memory access mode and near memory mode end, the NMA MC actively adjusts the memory state according to the recorded memory bank state change instructions to ensure consistency with the Host MC state, in order to switch Host Mode.

[0076] The explicit synchronization mechanism in step S1 prepares for switching to NMA Mode or Concurrent Mode, and the real-time update of the memory state recorded in the NMA MC further includes:

[0077] The DDR commands are transmitted via the command / address (C / A) bus. The NMA MC treats the received host commands as commands it issued and processes them accordingly, updating the memory state recorded in the NMA MC to ensure that it always tracks the latest memory state.

[0078] In this process, data is transferred directly between the host and memory devices via the data bus, bypassing the NMA's data buffer.

[0079] In step S2, the offloading-scheduling-return mechanism, the host memory controller offloading memory access requests to the near-memory accelerator memory controller further includes:

[0080] A new memory command is introduced for timing optimization of the unloading operation. The host memory controller sends an unloading access request to the near memory accelerator memory controller through the command / address bus, and various delays are omitted through timing optimization.

[0081] Specifically, the row activation delay tRCD and timing constraints tRTRs are omitted in the unloaded read command RDO. At the same time, the physical execution of the row activation instruction ACT is omitted in the unloaded read command RDO, and only its logical state is recorded.

[0082] Insert read operations into the unloaded write command WRO to omit write-to-read latency tWTR, read-to-write latency tRTW, and write latency tWL, thereby maximizing bus utilization.

[0083] In this embodiment, firstly, as Figure 6 As shown in (a), the timing constraints for consecutive RD (RDO) commands need to be maintained: tBL, for the occupancy of the data bus (DQ bus); tCCD_s or tCCD_l, for the BG and Bank states ( Figure 6 (Not shown in the text); tRTRs, used for switching between Ranks.

[0084] In this situation, one of the NMA memory controllers (such as NMA-1) will be idle, resulting in wasted bandwidth.

[0085] Based on the following two points, we can omit the corresponding timing constraints:

[0086] The RD command for unloading does not actually need to read data back from the Bank;

[0087] tRCD (corresponding to the delay for opening the Bank row) can also be omitted, because the unloaded ACT command does not actually change the Bank state.

[0088] like Figure 6 As shown in (b), this more efficient offloading method enables the NMA end to obtain higher aggregate bandwidth.

[0089] Secondly, such as Figure 6 As shown in (c), a similar bandwidth waste phenomenon can be observed during WR command unloading. However, timing constraints cannot be omitted in this case because writing data after issuing the WR command will occupy the data bus. To optimize this situation, if there are read requests (such as RDO commands) waiting for arbitration, we insert read accesses between the unloading write operations. This effectively omits tWTR (write-to-read latency) and tRTW (read-to-write latency) during the unloading process, and the write latency tWL can also be omitted.

[0090] like Figure 6As shown in (d), by scheduling more offload commands, the NMA memory controller can achieve higher bandwidth utilization. It should be noted that when a row switch occurs between WRO and RDO, the PREO and ACTO commands are required. Figure 6 (Not shown in the image).

[0091] Next, row locality between unloading commands can be naturally preserved within the FIFO structure. Therefore, host commands and NMA requests are isolated in the CMD FIFO and REQ FIFO of each bank in the NMA MC, respectively. However, arbitrary switching between these FIFOs can significantly reduce bandwidth utilization because they typically share little row locality. Furthermore, unloading commands generated based on potentially incorrect memory states in the HostMC, if issued directly by the NMA MC, may conflict with the actual bank state. To address these challenges, we introduce a Switch-Recovery (SR) method.

[0092] In step S2, the NMA MC schedules the host offloading access request and the NMA local access request according to the preset switch-recovery (SR) method, including:

[0093] In NMA MC, a separate command queue (CMD FIFO) and request queue (REQ FIFO) are set up for each memory bank. Host commands and near-memory accelerator requests are isolated in the command queue (CMD FIFO) and request queue (REQ FIFO), respectively. The queues are switched only when one of the following conditions is met:

[0094] There are no remaining commands or requests in the current queue;

[0095] Issue the PRE command.

[0096] Both conditions indicate that row locality can no longer be utilized in FIFO. The latter condition further ensures that switching has no overhead. In addition, since MC issues a PRE command when a row hit reaches a predetermined threshold, this method can prevent starvation in extreme cases (such as continuous access to the same row).

[0097] Then, by recording the last BSC command from the command queue CMD FIFO and the request queue REQ FIFO, it is determined whether recovery is needed, and the corresponding BSC command is issued during the switchover recovery to ensure the consistency of the memory state after the switchover.

[0098] In this embodiment, with Figure 7 Take a Bank as an example:

[0099] In phase ①, a PRE is issued from the CMD FIFO to update the BSC CMD and Bank status recorded in the SR unit;

[0100] In phase ②, due to the issuance of PRE, MC switches to REQ FIFO. Then, one ACT and three RD / WR are generated to complete all requests;

[0101] In phase ③, the NMA MC switches and performs recovery. The SR unit checks the switch recovery table to issue the corresponding BSC command;

[0102] In phase ④, commands in the CMD FIFO can be safely issued;

[0103] In stage ⑤, when the final PRE is issued, the N2H implicit synchronization is completed automatically.

[0104] Furthermore, in step S2, the time-division multiplexing interrupt includes:

[0105] Interrupt-based signals are used to notify the Host MC from the NMA MC, and the effective period for each memory column Rank to issue interrupt signals is limited by time-division multiplexing of the interrupt signals to reduce polling latency.

[0106] The adaptive batch processing mechanism further includes:

[0107] Interrupts are issued in the order in which memory accesses are unloaded, and data is returned in batches when the Host MC requests the NMA MC to return data. The batch size is equal to the number of interrupts that unloaded the read operation.

[0108] Subsequent interrupts must not be triggered if an earlier interrupt has not been completed.

[0109] Multiple read access interrupts are merged into a batch, and the data is returned in batches via a single return instruction (RT) command. The delay of the RT command is dynamically adjusted by the batch size and is executed before other memory access commands.

[0110] Specifically, similar to existing NMP designs, we use interrupt-based signals (such as a one-bit alert_n signal) to notify the Host MC from the NMA MC. However, this signal, like the C / A bus and DQ bus, is shared by multiple Ranks. Once the alert_n signal is issued, the Host MC needs to poll all Ranks to check the signal source. This polling latency (up to 50-80 cycles) is acceptable in accelerated designs (such as execution completion notifications), but unacceptable for memory access.

[0111] To address this challenge, we propose a simple yet effective method: time-division multiplexing of the interrupt signal to limit the effective period of the alert_n signal emitted by each Rank. Since the interface clocks of the Host MC and DIMM are synchronized during the calibration phase, the Host MC can determine the source Rank based on the arrival period of the alert_n signal. This method introduces only an average delay of NRank / 2 periods while completely eliminating polling overhead.

[0112] A key issue is that, given out-of-order execution between banks, a single signal cannot notify the Host MC which unloaded memory accesses have completed. Therefore, we restrict interrupts to be issued in the order in which memory accesses are unloaded. If an earlier access's interrupt has not yet been issued, an interrupt cannot be issued even if a newer access has completed. When the Host MC requests data from the NMA MC, it may have already accepted multiple interrupts. Therefore, data reads can be returned collectively in batches, with the batch size equal to the number of unloaded read interrupts.

[0113] We further designed the timing constraints for RT: unlike existing commands, its latency is not fixed, but determined by the batch size.

[0114] like Figure 8 As shown, the first RT command sent to NMA-0 takes tBL×2 time on the data bus to return data. The second RT command sent to NMA-1 takes tBL time. We replace the tRL latency with NMA index time because the returned data comes from the NMA buffer rather than the memory bank. RT can be viewed as one or more special RD commands that do not access the memory bank. In this way, it can coexist with existing DDR commands on the memory bus without violating the protocol.

[0115] Second Embodiment

[0116] This embodiment provides a heterogeneous execution system of main controller and near-memory accelerator in a near-memory computing architecture, such as Figure 9 As shown, it includes a Host Memory Controller (HMC) and a Near Memory Accelerator Memory Controller (NMA MC). The Host MC includes an offload table, a return unit, and a command encoding module. The NMA MC includes a bypass path, a configurable FIFO queue, and a switch-recovery unit.

[0117] The Offload Table (OT) is used to record the number of commands that have been uninstalled for each memory bank. When the number of commands reaches a preset threshold, the generation of new commands is paused.

[0118] The return unit (RU) is used to manage the timing and priority of the returned data and is configured to dynamically adjust the burst length based on the interrupt count;

[0119] The command encoding module uses reserved fields to encode the return instruction RT command and distinguishes the unload command by reusing the DDR command signal.

[0120] Specifically, in this embodiment, we use a unified design to implement concurrent mode and NMA mode. Each memory column Rank uses a one-bit register to indicate whether the mode is active. This register can also be set by an interrupt. An Offload Table (OT) is introduced to record the number of commands that have been offloaded for each Bank.

[0121] When the number of commands reaches a threshold, OT will prevent the generation of new commands. Figure 9 ①). The threshold for concurrent mode is set to the size of NMAFIFO, and the threshold for NMA mode is set to 0.

[0122] Commands generated from the request buffer bypass the scheduler. Figure 9 ②) To omit the corresponding timing constraints. After unloading, the commands will update the FSMs, scheduler, and arbitrator as if they were issued normally. Figure 9 ③).

[0123] Data return processing includes:

[0124] The Return Unit (RU) sends RT to retrieve the returned data;

[0125] Burst length equals the interrupt count from read accesses ( Figure 9 ④);

[0126] RT only needs to update the bus occupancy information recorded in the arbitrator;

[0127] The RT command is set to issue a higher priority than other DDR commands to ensure low access latency.

[0128] The return unit records information about unloaded memory accesses in Rank order, including:

[0129] The recorded information fields include physical address, read / write flag, and number of commands and their encoding method;

[0130] The operating mechanism includes:

[0131] These two-bit registers are used to update the OT (Operation Timeout) in real time when a new access is added or an old access is deleted. Figure 9 ⑤);

[0132] For write access, deletion occurs when a corresponding interrupt is received.

[0133] For read access, it is necessary to wait for both interruption and return data. At the same time, when the address and data are valid, they can be inserted into the read data buffer and then respond to higher-level memory requests.

[0134] The command encoding module encodes the new command using the current C / A signal interface. Figure 9 ⑥). As shown in the table below:

[0135]

[0136] In the NMA MC design,

[0137] The bypass path is used to directly pass commands from the host memory controller to the memory device in host mode, while updating the local memory state.

[0138] Specifically, in host mode:

[0139] Host commands are sent directly to the memory bank via a bypass path. Figure 9 ⑦);

[0140] These commands are then transmitted to FSMs, schedulers, and arbitrators to update the memory state and achieve explicit synchronization.

[0141] Similarly, writing or reading data is done directly between the memory bank and the data buffer, bypassing the data buffer.

[0142] The configurable FIFO queue is used to dynamically allocate FIFO queue capacity in concurrent mode, store access requests from host offloading and local access requests from near-memory accelerators respectively, and realize queue switching through an arbitrator.

[0143] In this embodiment, under concurrent mode:

[0144] The uninstallation command is inserted into the CMD FIFO via the CMD decoder and distributor. Figure 9 ⑧);

[0145] Instead of introducing a separate dedicated FIFO, AsyncDIMM rebuilds the existing REQ FIFO into a configurable one.

[0146] The switching-recovery unit is used to record BSC commands and generate recovery instructions to ensure the correctness of memory state and mode switching.

[0147] Specific implementation:

[0148] In NMA mode: NMA MC uses the entire REQ FIFO to store NMA requests;

[0149] In concurrent mode: half of the FIFO capacity is reused for host commands. Figure 9 ⑨). Since the command width is smaller than the request width, there is no need to modify the FIFO width. A host-NMA arbitrator is introduced for each hybrid FIFO unit to arbitrate between the two FIFOs. The SR unit records BSC commands and checks whether a command needs to be generated to restore the Bank state. Host commands are sent directly to the scheduler without checking the Bank state through the FSMs. This design avoids modifying the logic in the FSMs to support more diverse inputs.

[0150] The return unit in the NMA MC is similar to that in the Host MC, adding a return unit to record the unloading access status. Figure 9 ⑩). When the earliest access is completed, send an interrupt to the Host MC.

[0151] The return unit further includes a physical address mapping module and a priority controller, wherein,

[0152] The physical address mapping module is used to associate interrupt signals with corresponding memory accesses;

[0153] The priority controller is used to set the issuance priority of the return instruction RT command to be higher than that of other DDR commands;

[0154] The capacity allocation of the configurable FIFO queue is achieved in the following way:

[0155] In near-memory mode, the near-memory accelerator memory controller uses the entire request queue REQ FIFO to store near-memory accelerator local access requests. In concurrent mode, 50% of the FIFO queue capacity is used to store host offload commands.

[0156] The alternating scheduling of host offload commands and local access requests is achieved through an arbitrator in each hybrid FIFO unit.

[0157] Finally, it should be noted that the above description is only a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be pointed out that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

[0158] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for heterogeneous execution of a main controller and a near-memory accelerator in a near-memory computing architecture, characterized in that, The method comprises the following steps: S1: In the host mode, the host memory controller directly sends a DDR command to the memory device, and simultaneously bypasses the command to the near-memory accelerator memory controller, and updates the memory state recorded in the near-memory accelerator memory controller in real time; S2: In the simultaneous memory access mode and the near-memory mode, the host memory controller unloads the memory access request to the near-memory accelerator memory controller through an unloading mechanism, the near-memory accelerator memory controller schedules the access request unloaded by the host and the local access request of the near-memory accelerator according to a preset switching-recovery method, ensures the correctness of the memory state and the optimization of the row locality, and the near-memory accelerator memory controller returns data to the host memory controller through a time division multiplexing interrupt and an adaptive batch processing mechanism; S3: At the end of the simultaneous memory access mode and the near-memory mode, the near-memory accelerator memory controller actively adjusts the memory state according to the recorded memory bank state change instruction, ensures the consistency with the state of the host memory controller, and is used for switching of the host mode.

2. The method of claim 1, wherein the host processor and the near-memory accelerator are heterogeneous. In step S1, the real-time updating of the memory state recorded in the near-memory accelerator memory controller further comprises: The DDR command is transmitted through a command / address bus, the near-memory accelerator memory controller processes the received host command as a command issued by itself, updates the memory state recorded in the near-memory accelerator memory controller, and makes the near-memory accelerator memory controller always track the latest memory state.

3. The method of claim 1, wherein the host processor and the near-memory accelerator are heterogeneous. In step S2, the host memory controller unloading the memory access request to the near-memory accelerator memory controller further comprises: A new memory command is introduced for timing optimization of the unloading operation, the host memory controller sends the unloading access request to the near-memory accelerator memory controller through a command / address bus, and each delay is omitted through timing optimization.

4. The method of claim 3, wherein the host processor and the near-memory accelerator are heterogeneous. The timing optimization further comprises: In the unloading read command RDO, the row activation delay tRCD and the timing constraint tRTRs are omitted, and for the unloading read command RDO, the physical execution of the row activation instruction ACT is omitted, and only the logical state is recorded; In the unloading write command WRO, a read operation is inserted to omit the write-to-read delay tWTR, the read-to-write delay tRTW and the write delay tWL, so as to maximize the bus utilization.

5. The method of claim 1, wherein the host processor and the near-memory accelerator are heterogeneous execution units in a near-memory computing architecture. In step S2, the near-memory accelerator memory controller scheduling the access request unloaded by the host and the local access request of the near-memory accelerator according to the preset switching-recovery method further comprises: In the near-memory accelerator memory controller, an independent command queue CMD FIFO and a request queue REQ FIFO are set for each memory bank, the host command and the near-memory accelerator request are isolated in the command queue CMD FIFO and the request queue REQ FIFO, and the queue is switched only when one of the following conditions is met, comprising: There is no remaining command or request in the current queue; A precharge instruction PRE command is issued; By recording the last BSC command from the command queue CMD FIFO and the request queue REQ FIFO, which is an instruction for changing the state of a memory bank, to determine whether recovery is needed, and issuing a corresponding BSC command when switching is recovered to ensure the consistency of the memory state after switching.

6. The method of claim 1, wherein the host processor and the near-memory accelerator are heterogeneous execution units in a near-memory computing architecture. In step S2, the time-division multiplexed interrupt further includes: Using an interrupt-based signal to let the near-memory accelerator memory controller notify the host memory controller, and by time-division multiplexing the interrupt signal, limiting the valid period of the interrupt signal issued by each memory rank, to reduce the polling delay.

7. The method of claim 1, wherein the host processor and the near-memory accelerator are heterogeneous. In step S2, the adaptive batch mechanism further includes: Limiting the issuance of interrupts in the order of memory access offloading, and collectively returning data in batches when the host memory controller requires the near-memory accelerator memory controller to return data, the batch size being equal to the number of offloaded read interrupts, wherein, Subsequent interrupts are prohibited from triggering when an earlier interrupt is not completed; Combining multiple read access interrupts into batches, and returning data in batches through a single return instruction RT command, the delay of the RT command being dynamically adjusted by the batch size, and the RT command being executed in priority to other memory access commands.

8. A host in a near-memory computing architecture implementing the method of any of claims 1-7, wherein the host is configured to: A system including a host memory controller and a near-memory accelerator memory controller, the host memory controller including an offloading table, a return unit, and a command encoding module, and the near-memory accelerator memory controller including a bypass path, a configurable FIFO queue, and a switching-recovery unit, wherein The offloading table is used to record the number of offloaded commands for each memory bank, and to pause the generation of new commands when a preset threshold is reached; The return unit is used to manage the timing and priority of returning data, and is configured to dynamically adjust the burst length according to the interrupt count; The command encoding module encodes the return instruction RT command using a reserved field, and distinguishes offloaded commands by multiplexing DDR command signals; The bypass path is used to directly transmit the commands of the host memory controller to the memory device in the host mode, while updating the local memory state; The configurable FIFO queue is used to dynamically allocate the capacity of the FIFO queue in the concurrent mode, to store the access requests offloaded by the host and the local access requests of the near-memory accelerator respectively, and to realize queue switching through an arbitrator; The switching-recovery unit is used to record BSC commands and generate recovery instructions to ensure the correctness of the memory state and the mode switching.

9. The host and near-memory accelerator heterogeneous execution system in a near-memory computing architecture of claim 8, wherein, The return unit further includes a physical address mapping module and a priority controller, wherein The physical address mapping module is used to associate the interrupt signal with the corresponding memory access; The priority controller is used to set the priority of the return instruction RT command to be higher than that of other DDR commands; The capacity allocation of the configurable FIFO queue is realized in the following way: In the near-memory mode, the near-memory accelerator memory controller uses the entire request queue REQ FIFO to store the local access requests of the near-memory accelerator, and in the concurrent mode, 50% of the capacity of the FIFO queue is used to store the host offloaded commands; Alternating scheduling of host offloaded commands and local access requests is achieved by an arbiter in each hybrid FIFO unit.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the method for heterogeneous execution of a host and a near-memory accelerator in a near-memory computing architecture as claimed in any one of claims 1-7.

11. An electronic device, comprising: comprising: one or more processors; a memory device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method for heterogeneous execution of a host and a near-memory accelerator in a near-memory computing architecture as claimed in any one of claims 1-7.

Images