DDR4 / 5 bank-based matrix vector multiplication implementation method and system

Abstract

This invention relates to the fields of in-memory computing and memory controller technology, and discloses a matrix-vector multiplication implementation method and system based on DDR4 / 5 banks. The method includes: writing matrix data and input vector data into multiple cascaded memory banks in DDR memory, wherein the DDR memory is standard DDR4 or DDR5 memory; generating and sending a sequence of computation commands to the DDR memory to schedule the multiple memory banks to perform cascaded computation; and obtaining the final computation result from the memory bank that is the last stage in the cascaded computation. This invention cleverly maps the computation process to the cascaded structure of DDR memory memory banks, achieving high-energy-efficiency in-memory computing or near-memory computing, effectively improving energy efficiency and computational efficiency, and reducing data transfer overhead.

Description

Technical Field

[0001] This invention relates to the fields of in-memory computing and memory controller technology, and more specifically, to a matrix-vector multiplication implementation method and system based on DDR4 / 5Bank. Background Technology

[0002] In the current era of rapid development in artificial intelligence, the Internet of Things, and 5G technologies, large-scale AI applications characterized by tens or even trillions of parameters have become a core driving force. However, the traditional von Neumann computing architecture, due to the physical separation of computing and storage units, leads to frequent data transfer between the two, forming the so-called "memory wall" bottleneck. Research shows that in traditional architectures, over 60% of system energy consumption and a significant portion of latency are not used for actual computation but are consumed in data transfer. This "storage-computation-transfer" bottleneck severely restricts the improvement of overall system performance, especially in large model inference scenarios, where matrix-vector multiplication, as a key operation in the decoding stage, directly determines end-to-end inference latency and throughput.

[0003] However, current DRAM in-memory computing architectures still face significant challenges in implementing large-scale matrix-vector multiplication. How to fully unleash the high bandwidth potential of existing commercial DDR4 / DDR5 memory standards through architectural innovation, and efficiently and stably implement large-scale matrix-vector multiplication without modifying DRAM hardware, to solve the aforementioned data transfer bottlenecks, uneven resource utilization, and limited computational efficiency, has become a crucial technical challenge that urgently needs to be addressed in this field. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a matrix-vector multiplication implementation method and system based on DDR4 / 5Bank, aiming to solve the "memory wall" bottleneck and high data transfer overhead problems of the traditional von Neumann architecture. The technical solution of the present invention is as follows: In a first aspect, the present invention provides a matrix-vector multiplication implementation method based on DDR4 / 5Bank, comprising: Matrix data and input vector data are written to multiple memory banks that are cascaded in sequence in DDR memory, where the DDR memory is standard DDR4 memory or DDR5 memory; Generate and send a sequence of computation command to DDR memory to schedule multiple memory banks to perform cascaded computations; The calculation command sequence includes: Send a first command to the storage group that is the previous stage in the cascaded computation, so that it generates the computation result of the previous stage and sends the computation result of the previous stage to the storage group that is the next stage. After a preset column operation delay time, a second command is sent to the next level of storage bank to generate the next level calculation result and add it to the received previous level calculation result; The final computation result is obtained from the memory bank group that is the last stage in the cascaded computation.

[0005] The above solution solves the "memory wall" bottleneck and high data transfer overhead of the traditional von Neumann architecture, and improves the computational efficiency of matrix-vector multiplication and system performance.

[0006] Furthermore, the present invention also proposes a method for implementing matrix-vector multiplication based on DDR4 / 5-Bank, comprising the following steps: writing matrix data and input vector data into multiple cascaded memory banks in DDR memory. The memory controller writes the matrix data sub-blocks to be calculated and the input vector data into multiple memory banks that are cascaded in DDR memory according to a preset address mapping strategy.

[0007] The above solution optimizes the storage and access efficiency of data in DDR memory, further improving computing performance.

[0008] Furthermore, the present invention also proposes a matrix-vector multiplication implementation method based on DDR4 / 5Bank, wherein the preset address mapping strategy includes: decomposing matrix data sub-blocks into multiple consecutive data sub-blocks according to a preset decomposition logic, and mapping the multiple consecutive data sub-blocks and their corresponding input vector data sequentially to multiple memory banks cascaded in DDR memory.

[0009] Furthermore, the present invention also proposes that, according to a matrix-vector multiplication implementation method based on DDR4 / 5Bank, the calculation command sequence further includes sending activation commands to multiple memory banks before sending the first command, so that the corresponding row buffers are in a ready state.

[0010] Furthermore, the present invention also proposes a method for implementing matrix-vector multiplication based on DDR4 / 5Bank, wherein the step of sending a first command to the memory bank group serving as the preceding stage in cascaded computation to generate the computation result of the preceding stage includes: Based on the first command received, the weight matrix data and input vector data are read from the storage array of the previous storage group, and the results of the previous calculation are generated by using peripheral circuits for analog calculation or by performing multiplication operations through digital logic.

[0011] Furthermore, the present invention also proposes a method for implementing matrix-vector multiplication based on DDR4 / 5Bank, wherein the step of sending the calculation result of the previous stage to the memory bank group that serves as the next stage includes: The results of calculations in the previous stage are passed to the memory bank in the next stage through a cascading path.

[0012] Furthermore, this invention also proposes a matrix-vector multiplication implementation method based on DDR4 / 5Bank, in which the method further includes the following steps during the cascaded computation process: Perform diagnostic operations periodically and measure the actual physical execution time of the diagnostic operations; Based on the deviation between the actual physical execution time and the preset theoretical execution time, the target count value of the clock cycle counter corresponding to the column operation delay time is dynamically adjusted to ensure that the column operation delay time meets the predefined column operation interval between memory banks.

[0013] Furthermore, the present invention also proposes a matrix-vector multiplication implementation method based on DDR4 / 5Bank, wherein the diagnostic operation includes a combination of read and write commands sent to the same or different memory banks and with a fixed execution time.

[0014] Furthermore, this invention also proposes a matrix-vector multiplication implementation method based on DDR4 / 5Bank, which includes the following steps after obtaining the final calculation result: The matrix-vector multiplication calculation is re-executed using the memory controller based on the original weight matrix and input vector data to obtain the verification calculation results; The verification calculation results are compared with the final calculation results; If the comparison results are inconsistent, the calculation command sequence will be resent or the weight matrix and input vector data will be written into a preset set of backup storage units and the preset set of backup storage units will be scheduled to perform cascade calculations.

[0015] Secondly, this invention also proposes a matrix-vector multiplication implementation system based on DDR4 / 5Bank, comprising: The memory controller is used to write matrix data and input vector data into multiple storage banks that are cascaded in DDR memory, where the DDR memory is standard DDR4 or DDR5 memory; it is also used to generate and send a sequence of computation commands to the DDR memory to schedule multiple storage banks to perform cascaded computation; and it is also used to obtain the final computation result from the storage bank that is the last stage in the cascaded computation. DDR memory is used to receive sequences of computation commands and schedule multiple memory banks to perform cascaded computations. The calculation command sequence includes: Send a first command to the storage group that is the previous stage in the cascaded computation, so that it generates the computation result of the previous stage and sends it to the storage group that is the next stage. After a preset column operation delay time, a second command is sent to the next level of storage bank to generate the next level calculation result and add it to the received previous level calculation result.

[0016] The above scheme provides a hardware system for implementing the above method, which is easy to deploy and apply in practice and has good feasibility.

[0017] In summary, this invention provides a matrix-vector multiplication method and system based on DDR4 / 5-bank memory. The method involves writing matrix data and input vector data into multiple cascaded memory banks in DDR memory, generating a sequence of computational commands to schedule these memory banks for cascaded computation. The computational command sequence includes sending a first command to the preceding memory bank to generate the result of the preceding computation and sending it to the next memory bank; and sending a second command to the next memory bank after a preset column operation delay time to generate the result of the next computation and adding it to the received result of the preceding computation. Finally, the final computational result is obtained from the last memory bank. This method solves the "memory wall" bottleneck and high data transport overhead problems of traditional von Neumann architectures, improving the computational efficiency and system performance of matrix-vector multiplication. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a matrix-vector multiplication method based on DDR4 / 5Bank provided in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a matrix-vector multiplication implementation system based on DDR4 / 5Bank, provided as an embodiment of the present invention.

[0020] Labeling explanation: 210, memory controller; 220, DDR memory. Detailed Implementation

[0021] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of this invention. The components of this invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] In traditional architectures, data is frequently transferred between the processor and memory, resulting in significant time latency and energy consumption. This is especially true in large-scale artificial intelligence applications, where the performance of matrix-vector multiplication, a core computation, is directly limited by the bottleneck of data transfer.

[0024] Firstly, please see Figure 1 This invention proposes a matrix-vector multiplication implementation method based on DDR4 / 5Bank, which includes: S1. Write the matrix data and input vector data into multiple memory banks that are cascaded in sequence in the DDR memory 220, where the DDR memory 220 is a standard DDR4 memory or DDR5 memory; S2. Generate and send a sequence of computation command to DDR memory 220 to schedule multiple memory banks to perform cascaded computation; The calculation command sequence includes: Send a first command to the storage group that is the previous stage in the cascaded computation, so that it generates the computation result of the previous stage and sends the computation result of the previous stage to the storage group that is the next stage. After a preset column operation delay time, a second command is sent to the next level of storage bank to generate the next level calculation result and add it to the received previous level calculation result; S3. Obtain the final calculation result from the storage group that is the last stage in the cascaded calculation.

[0025] Specifically, the working principle of this method can be understood as a process of building an internal data processing pipeline on standard hardware. The starting point of the entire process is data initialization and layout. When a matrix-vector multiplication task is triggered, the main system first writes the weight matrix to be calculated and the input vector data to be input into the DDR memory 220 in a conventional manner through the memory controller 210. However, the writing process is not random, but follows an address mapping strategy tailored for subsequent in-memory computation. This strategy ensures that data that logically needs to be computed together is physically and precisely distributed into different bank groups within the DDR memory 220. In the DDR4 or DDR5 memory standards, a bank group is a collection of highly parallel storage units within the memory chip. By logically decomposing a large-scale matrix into multiple sub-blocks and storing these sub-blocks interleaved in different bank groups, the necessary conditions are created for subsequent parallel computation and data cascading.

[0026] After the data layout is completed, the computation process is triggered and driven by a series of specially orchestrated memory operation commands issued by the memory controller 210. This command sequence is not a traditional read / write instruction, but a computational program capable of scheduling multiple memory banks to work collaboratively. At the start of the computation, the memory controller 210 first sends activation commands to all participating memory banks. This step loads the matrix and vector data stored in the memory array into the respective row buffers of each memory bank, also known as inductive amplifiers. The row buffer is a temporary storage area before data is read or written; pre-activation ensures that the data is in a ready state, preparing it for subsequent rapid computation.

[0027] Next, the cascaded computation officially begins. Memory controller 210 sends a first command to the memory bank group that serves as the first stage of the cascaded computation, such as memory bank group 0. This first command may appear to be a standard read command, but its true intention is not to read data back to memory controller 210. Instead, the command triggers peripheral circuitry within memory bank group 0, such as inductive amplifiers, to perform local multiplication or multiply-add operations on the matrix sub-block data in the row buffer and the input vector data, utilizing their analog characteristics or integrated simple digital logic, thereby generating a result of the previous stage's computation, i.e., a partial sum.

[0028] The key step lies in the transmission of the result of this previous stage calculation. After generation, the result is not returned to the memory controller 210 via the external memory bus, but is instead driven to a global data bus or a specific cascaded path within the DDR memory 220 chip. Next, the memory controller 210 waits precisely for a preset column operation latency. This latency, corresponding to the minimum time interval (tCCD_S) for consecutive column accesses between different memory banks in the DDR4 / 5 standard, is extremely short. Utilizing this timing characteristic is crucial for achieving efficient internal data transfer.

[0029] After the tCCD_S delay ends, the memory controller 210 immediately sends a second command to the next stage of the cascaded computation, such as memory bank 1. This second command may appear to be a standard write command, but its function is also redefined. This command causes the input / output circuitry of memory bank 1 to capture the computation result from the previous stage, passed from memory bank 0, from the internal data bus and accumulate it with the data in memory bank 1's own line buffer. In some implementations, memory bank 1 may also perform its own local computation first, and then add the result to the received result from the previous stage.

[0030] This process—generating local results, internal transfer, precise delay, receiving and accumulating—is carried out sequentially among multiple memory banks, much like a pipeline. After memory bank 1 completes its accumulation, it passes the new accumulated result to memory bank 2 in the same way, which then passes it to memory bank 3, and so on, until the last memory bank participating in the calculation.

[0031] After the cascaded computation pipeline has passed through all designated memory banks, the final, complete matrix-vector multiplication result is accumulated in the row buffer of the last memory bank. At this point, the memory controller 210 only needs to send a regular read command to this last memory bank to read the final calculation result from the DDR memory 220 and transmit it to the host system.

[0032] In this way, most of the calculations and intermediate data transfers are completed inside the DDR memory 220 chip, greatly reducing the number of round trips between memory and processor, thus effectively overcoming the memory wall bottleneck and achieving high efficiency and low power consumption in the calculation process.

[0033] The core idea of ​​the technical solution of the present invention is to utilize the internal architecture characteristics of existing commercial standard DDR4 or DDR5 memory, and through functional enhancement and ingenious arrangement of command sequences of memory controller 210, to achieve an efficient in-memory computing mode without modifying the dynamic random access memory hardware, thereby significantly reducing data transfer overhead and improving the overall computational efficiency and energy efficiency of matrix-vector multiplication.

[0034] Furthermore, in some embodiments, the step of writing matrix data and input vector data into multiple memory banks cascaded sequentially in the DDR memory 220 includes: The memory controller 210 writes the matrix data sub-blocks to be calculated and the input vector data into multiple memory banks cascaded in sequence in the DDR memory 220 according to a preset address mapping strategy.

[0035] Specifically, this step is the cornerstone of the entire in-memory computing scheme. The memory controller 210, as the main entity responsible for data writing, is strictly guided by a preset address mapping strategy. The core objective of this strategy is to break the principle of locality of reference in conventional memory access, physically distributing large blocks of logically contiguous matrix data across different memory banks. For example, a default address mapping scheme might prioritize filling all rows within a single memory bank with contiguous memory addresses, then other memory banks within the same memory bank group, and finally switching to the next memory bank group. This approach leads to computational tasks being densely concentrated within a few memory bank groups, failing to effectively utilize the multi-memory bank parallel capabilities of the DDR memory 220, and also failing to trigger efficient cross-memory bank data transfer.

[0036] Therefore, the preset address mapping strategy interprets or remaps certain bits of the physical address in a special way. By inserting the address bits normally used to select memory bank groups into the middle of the address bits that determine row or column addresses, an interleaved or staggered mapping can be achieved. In this way, when the memory controller 210 writes a large matrix sequentially, the linear growth of the logical address periodically changes the physical address bits used to select memory bank groups, thereby ensuring that consecutive data sub-blocks of the matrix are written evenly and in a rotating manner to different memory bank groups.

[0037] Furthermore, in some embodiments, the preset address mapping strategy includes: The matrix data sub-blocks are decomposed into multiple consecutive data sub-blocks according to the preset decomposition logic, and the multiple consecutive data sub-blocks and their corresponding input vector data are sequentially mapped to multiple memory banks that are cascaded in DDR memory 220.

[0038] To illustrate this process more clearly, consider a specific implementation scenario. Assume a weight matrix W to be calculated has a dimension of 1024 x 1024, and an input vector V has a dimension of 1024 x 1. The system uses DDR4 memory with four memory banks, labeled BG0, BG1, BG2, and BG3.

[0039] According to the preset decomposition logic, the memory controller 210 first logically decomposes the 1024 x 1024 matrix W into four consecutive data sub-blocks, each with a dimension of 256 x 1024. These four sub-blocks are denoted as W0, W1, W2, and W3, respectively.

[0040] Subsequently, the memory controller 210 begins to perform address mapping and data writing.

[0041] The first step is to map and write the first matrix sub-block W0 and the complete input vector V into the specified row address space of the memory group BG0.

[0042] The second step is to map and write the second matrix sub-block W1 and the complete input vector V into the specified row address space of the memory group BG1.

[0043] The third step is to map and write the third matrix sub-block W2 and the complete input vector V into the specified row address space of the memory block BG2.

[0044] The fourth step is to map and write the fourth matrix sub-block W3 and the complete input vector V into the specified row address space of the memory block BG3.

[0045] In this way, logically adjacent computational tasks—W0 multiplied by V, W1 multiplied by V, W2 multiplied by V, and W3 multiplied by V—are physically deployed across four different memory banks that can be efficiently connected via the tCCD_S timing sequence. This data layout paves the way for cascaded computations triggered by subsequent command sequences, ensuring that the computational pipeline can flow smoothly between BG0, BG1, BG2, and BG3.

[0046] Furthermore, in some embodiments, the computation command sequence further includes sending activation commands to multiple memory banks before sending the first command, so that the corresponding row buffers are in a ready state.

[0047] This step further optimizes the computation process, aiming to reduce latency during computation startup. In the DDR 220 memory operating specifications, any access to data in the storage array must first execute an activation command to completely read the data of the target row into the corresponding row buffer of that memory bank. This activation operation itself requires a non-negligible time, known as row activation latency. If activation of the corresponding memory bank group only begins after the first command triggering the computation is issued, the startup of the entire computation pipeline will be slowed down by this latency.

[0048] Therefore, in an optimized implementation, the memory controller 210 broadcasts or sequentially sends activation commands to all memory banks (BG0, BG1, BG2, and BG3) that will participate in the computation before generating and sending the core computation command sequence. These activation commands specify the row addresses of the stored matrix sub-blocks and vector data. Upon receiving the commands, each memory bank loads the required data from its respective memory array into the row buffer in parallel. Once this pre-activation process is complete, all data participating in the computation is ready. Sending the first command at this point to initiate the cascaded computation enables seamless connection of computational operations, thereby maximizing the execution efficiency of the entire process.

[0049] Furthermore, in some embodiments, the step of sending a first command to the memory bank group serving as the preceding stage in the cascaded computation to produce the computation result of the preceding stage includes: Based on the first command received, the weight matrix data and input vector data are read from the storage array of the previous storage group, and the results of the previous calculation are generated by using peripheral circuits for analog calculation or by performing multiplication operations through digital logic.

[0050] This feature reveals the core mechanism by which computation is implemented within standard DDR memory 220. When a preceding storage bank, such as BG0, receives the first command from memory controller 210, it performs a series of operations different from conventional data reading. Since the data has already been loaded into the row buffer via a pre-activation command, the first command can directly act on the data in the row buffer.

[0051] There are two main paths to implement the specific calculation.

[0052] The first approach utilizes external circuitry for analog calculations. The row buffer of DDR220 memory is essentially an array of numerous inductive amplifiers. These amplifiers amplify the weak charge on the memory cells during data reading, and their operation inherently involves analog signal processing. By controlling the timing of the read / write control signals, bitwise logical operations, such as AND, OR, and NOT, can be performed using the charging and discharging characteristics of the inductive amplifiers or the charge-sharing effect among multiple amplifiers. Combining these basic bitwise operations allows for indirect multiplication and addition. The advantage of this approach is its extremely high energy efficiency, as it reuses the existing analog circuitry in DDR220 memory with virtually no additional hardware overhead.

[0053] The second approach involves multiplication via digital logic. While this invention emphasizes not modifying the DRAM hardware, in some advanced in-memory computing research or future standards, a small number of low-power digital logic units, such as simple multipliers and adders, may be integrated within the periphery of the memory bank. In this case, the first command instructs the memory bank to route data in the line buffer to these digital logic units for precise multiplication, generating the result of the previous stage's computation.

[0054] Regardless of the path taken, the common point is that the calculation process occurs entirely within the memory bank or its immediate vicinity, and the resulting calculation results are temporarily stored in internal registers or line buffers, preparing for the next internal transfer.

[0055] Furthermore, in some embodiments, the step of sending the calculation result of the previous stage to the storage group that serves as the next stage includes: The results of calculations in the previous stage are passed to the memory bank in the next stage through a cascading path.

[0056] The cascading path here forms the physical basis for efficient data flow between memory banks. In a standard DDR220 memory chip design, all memory banks typically share an internal, high-bandwidth data bus for transmitting data within the chip and exchanging data with external input / output interfaces. The technical solution of this invention cleverly utilizes this internal data bus as the cascading path.

[0057] After memory bank BG0 generates the result of the previous stage's calculation, the subsequent part of the first command issued by memory controller 210, or a microinstruction that follows, controls the output driver of BG0 to drive this calculation result onto the internal data bus. Simultaneously, the output drivers of all other memory banks are in a high-impedance state to avoid bus conflicts. At this point, the data signal of the previous stage's calculation result is broadcast on the chip's internal global bus. Due to the extremely short transmission distance and the fact that the process occurs entirely within the chip, the latency is extremely low, and the power consumption is far less than that of data transfer via external memory channels.

[0058] In some more advanced implementations, the cascading path may also be a dedicated, point-to-point or optimized data path designed specifically for communication between memory banks to further reduce latency and power consumption. However, utilizing the existing internal global data bus is the implementation method with the best compatibility with standard DDR hardware.

[0059] To support the relatively abstract technical solution of cascade computing, two specific implementations are provided below, demonstrating two different cascade computing modes: linear and tree-like.

[0060] Example 1: Linear Cascaded Computation Mode.

[0061] This mode is suitable for most medium-sized matrix-vector multiplications, with a clear data flow path and relatively simple control logic. Continuing with the previous example of a 1024-by-1024 matrix and four memory blocks, the data has been stored in BG0 to BG3 in the order W0 to W3.

[0062] The calculation process is as follows: First, memory controller 210 sends the first command to BG0. BG0 performs the calculation of W0 multiplied by V to obtain a partial sum S0. Subsequently, S0 is driven onto the internal cascade path.

[0063] In the second step, after a delay of tCCD_S, the memory controller 210 sends a second command to BG1. BG1 first performs the calculation of W1 multiplied by V, obtaining a local result P1. Then, BG1 captures S0 from the cascade path and accumulates P1 with S0 to obtain a new partial sum S1. Subsequently, S1 is driven onto the internal cascade path.

[0064] Third, after a delay of tCCD_S, memory controller 210 sends a second command to BG2. BG2 performs the calculation of W2 multiplied by V, obtains a local result P2, captures S1 from the cascade path, and adds P2 and S1 together to obtain a partial sum S2. S2 is then driven onto the internal cascade path.

[0065] Fourthly, similarly, memory controller 210 sends a second command to BG3. BG3 performs the calculation of W3 multiplied by V, obtains the local result P3, captures S2 from the cascade path, and adds P3 and S2 together to obtain the final result vector Y.

[0066] Fifth step, memory controller 210 sends a standard read command to BG3 to read the result Y.

[0067] In this linear mode, data flows sequentially through BG0, BG1, BG2, and BG3, like on an assembly line, with each stage performing a calculation and accumulation, ultimately yielding the complete result at the last stage.

[0068] Example 2: Tree-like cascaded computing mode.

[0069] This mode is suitable for larger matrices or scenarios requiring maximum parallelism. Suppose we have a 2048 x 2048 matrix, and the system uses DDR5 memory with eight memory banks, labeled BG0 to BG7. The matrix is ​​decomposed into eight 256 x 2048 sub-blocks, W0 to W7, which are stored in BG0 to BG7 respectively.

[0070] The computational process is designed as a two-layer tree structure: First-level parallel computation: The memory controller 210 sends the first command to BG0, BG2, BG4, and BG6 simultaneously or at very short time intervals. These four memory banks execute their respective local calculations in parallel: BG0 calculates W0 multiplied by V to obtain S0, BG2 calculates W2 multiplied by V to obtain S2, BG4 calculates W4 multiplied by V to obtain S4, and BG6 calculates W6 multiplied by V to obtain S6.

[0071] At the same time, the memory controller 210 also sends activation commands to BG1, BG3, BG5, and BG7, putting them in a state ready to receive data.

[0072] First layer, intermediate accumulation: After a delay of tCCD_S, the memory controller 210 sends a second command to BG1, causing it to capture S0 from BG0 and accumulate it with the result of multiplying W1 by V to obtain the intermediate sum S01.

[0073] At the same time, the memory controller 210 sends a second command to BG3, causing it to capture S2 from BG2 and accumulate it with the result of multiplying W3 by V to obtain the intermediate sum S23.

[0074] Similarly, BG5 is summed with the result from BG4 to get S45; BG7 is summed with the result from BG6 to get S67.

[0075] At this point, the first layer of computation is complete, and the parallelism of the computation task is 4.

[0076] The final accumulation of the second layer: The memory controller 210 uses S01 and S23 as new inputs to drive the cascaded path. For example, S01 is first passed from BG1 to BG2.

[0077] After a delay of tCCD_S, the memory controller 210 sends a command to BG3 to capture S01 from BG2 and add it to its own S23 to obtain half of the final result, denoted as Y_upper.

[0078] In parallel, S45 and S67 are also accumulated between BG5 and BG7 to obtain the other half of the final result, denoted as Y_lower.

[0079] The final step is to merge Y_upper and Y_lower. This can be done in another free storage group within memory, or one can be read out and merged with the other in memory controller 210.

[0080] This tree-like structure significantly reduces critical path latency by grouping computational tasks for parallel processing. For large-scale computational tasks, its performance improvement may exceed that of the linear cascading mode.

[0081] However, the correctness of all the aforementioned efficient cascaded computing models highly depends on the precise control of the tCCD_S timing by the memory controller 210. In environments requiring long-term uninterrupted operation, such as computing centers, electronic components can experience performance degradation due to physical aging. For example, the equivalent series resistance of the filter capacitors in the power supply regulator module inside the memory controller 210 chip slowly increases over time. This causes slight but frequent drops in the core power supply voltage of the memory controller 210 during high-intensity computing tasks. Such voltage fluctuations interfere with the stability of the internal clock generator, causing a slight frequency drift or jitter in its output clock signal. The timing accuracy of the tCCD_S clock cycle counter, which relies on this clock signal for counting, is compromised, causing the actual physical time interval between commands issued by the memory controller 210 to deviate from the standard tCCD_S requirement. This small timing deviation is sufficient to cause signal conflicts or data latch errors on the data bus inside the DRAM chip, ultimately leading to deviations in the calculation results of the entire matrix-vector multiplication and affecting the reliability of the artificial intelligence service.

[0082] To address this issue, the present invention further proposes that, during the cascaded computation process, the method further includes: periodically performing diagnostic operations and measuring the actual physical execution time of the diagnostic operations; and dynamically adjusting the target count value of the clock cycle counter corresponding to the column operation delay time based on the deviation between the actual physical execution time and the preset theoretical execution time, so as to ensure that the column operation delay time meets the predefined inter-memory column operation interval.

[0083] This technical solution introduces an adaptive dynamic calibration mechanism. Its core idea is to no longer assume that the main clock of the memory controller 210 is absolutely stable, but rather to acknowledge and accommodate its potential drift. To this end, the memory controller 210 integrates an independent, high-precision physical time reference source, such as a crystal oscillator with an independent and stable power supply, whose timing accuracy is unaffected by fluctuations in the core power supply voltage.

[0084] During system operation, the memory controller 210 periodically initiates a diagnostic program, such as during task breaks or when the system load is low. This program sends a series of pre-designed combinations of read and write commands with fixed execution times to the DRAM chip. The execution time of these commands is independent of the tCCD_S timing, therefore their theoretical physical execution time is known and constant.

[0085] When executing these diagnostic commands, the memory controller 210 uses the independent physical time reference source to accurately measure the actual physical time consumed from the issuance to completion of the command sequence. This measured actual time is then compared to a preset theoretical time. If there is a discrepancy, it means that the main clock signal of the memory controller 210 has drifted. For example, if the actual time is less than the theoretical time, it means the main clock is running fast; conversely, it means the main clock is running slow.

[0086] Based on the calculated deviation or drift rate, the memory controller 210 immediately and dynamically adjusts the target count value of the clock cycle counter used for tCCD_S timing. Assuming the standard duration of tCCD_S corresponds to 10 clock cycles, if the diagnostics detect that the main clock is 1% fast, the memory controller 210 will adjust the target value of the tCCD_S counter from 10 to 10.1. Although the counter cannot count decimals, an equivalent extension can be achieved through finer logic, such as inserting wait states in certain cycles, to ensure that the final physical length of the wait is strictly equal to the standard tCCD_S duration. Conversely, if the main clock is slow, the target count value will be adjusted downwards accordingly.

[0087] Through this periodic measurement, comparison, and dynamic adjustment, even if the main clock of the memory controller 210 continues to drift due to component aging, the counter controlling the critical timing can always output timing results highly consistent with physical time. This ensures that the timing intervals of the cascaded computation command sequences between Bank Groups are always accurate, thereby fundamentally avoiding data transmission errors caused by timing issues and guaranteeing the long-term computational correctness of large-scale matrix-vector multiplication results.

[0088] Furthermore, in some embodiments, the diagnostic operation includes a combination of read and write commands sent to the same or different memory groups and executed at a fixed time.

[0089] Specifically, the design of diagnostic operations needs to balance measurement accuracy with low interference to normal operations. A feasible combination of diagnostic commands is to execute one write command and one read command consecutively for the same row address within the same memory bank. The theoretical execution time of this operation sequence is determined by a series of fixed timing parameters such as write recovery time, row precharge time, row activation time, and column read delay, and does not involve the variable tCCD_S. By measuring the actual time consumption of this fixed sequence, the frequency variation of the master clock can be effectively reflected.

[0090] Another diagnostic command combination could be to send two read commands to two different memory banks, spaced apart by a relatively long column operation delay time tCCD_L. tCCD_L is a well-defined and relatively lenient timing parameter in the DDR standard, with high tolerance and less susceptibility to errors caused by minor clock drift. Measuring the actual time interval between these two commands can also be used to calibrate the master clock. Choosing different diagnostic operation combinations allows for a more comprehensive assessment of the clock's stability under different operating conditions.

[0091] Even with precise timing control, data bit flipping can still occur during complex calculations due to cosmic rays or other transient disturbances, leading to computational errors. To ensure the absolute reliability of the final result, especially in applications with extremely high precision requirements such as medical diagnosis and autonomous driving, this invention further proposes a result verification and error recovery mechanism.

[0092] After obtaining the final calculation result, the method further includes: re-executing the matrix-vector multiplication calculation based on the original weight matrix and input vector data through the memory controller 210 to obtain the verification calculation result; comparing the verification calculation result with the final calculation result; if the comparison result is inconsistent, resending the calculation command sequence or writing the weight matrix and input vector data into a preset set of backup storage and scheduling the preset set of backup storage to perform cascade calculation.

[0093] This mechanism adds a layer of reliability assurance to in-memory computation. Once the final computation result is obtained from the last level of memory, it is not immediately adopted. The memory controller 210 or the main processor uses a traditional, though slower, but more reliable method, such as on the CPU core, to perform a complete matrix-vector multiplication again using the original weight matrix and input vector data, obtaining a verification result.

[0094] Subsequently, the final calculation result obtained from the in-memory computation is compared element-by-element with this verification calculation result. If the two are completely consistent or within the allowable error range, the in-memory computation result is confirmed to be valid.

[0095] If the comparison results are inconsistent, it indicates that an error may have occurred during the in-memory computation. In this case, the system will initiate an error recovery process. The first recovery strategy is retry, whereby the memory controller 210 simply resends the complete sequence of computation commands and executes the entire cascaded computation process again. This strategy is suitable for soft errors caused by transient interference, and retrying has a high probability of yielding a correct result.

[0096] If the result is still incorrect after retrying, or for greater robustness, the system can employ a second recovery strategy: failover. The system will activate a set of reserved, standby backup memory banks. The memory controller 210 will rewrite the original weight matrix and input vector data into these backup memory banks and then schedule this backup set to perform cascaded computation. This strategy effectively avoids memory banks that may have permanent or semi-permanent hardware failures, thus ensuring that the computational task can ultimately be completed correctly.

[0097] Secondly, see Figure 2 This invention also proposes a matrix-vector multiplication implementation system based on DDR4 / 5Bank, which serves as the physical carrier for implementing the aforementioned method. The system includes: The memory controller 210 is used to write matrix data and input vector data into multiple storage banks that are cascaded in sequence in the DDR memory 220, wherein the DDR memory 220 is standard DDR4 memory or DDR5 memory; it is also used to generate and send a sequence of calculation commands to the DDR memory 220 to schedule multiple storage banks to perform cascaded calculations; and it is also used to obtain the final calculation result from the storage bank that is the last stage in the cascaded calculation. DDR memory 220 is used to receive computation command sequences and schedule multiple memory bank groups to perform cascaded computations. The calculation command sequence includes: Send a first command to the storage group that is the previous stage in the cascaded computation, so that it generates the computation result of the previous stage and sends it to the storage group that is the next stage. After a preset column operation delay time, a second command is sent to the next level of storage bank to generate the next level calculation result and add it to the received previous level calculation result.

[0098] The memory controller 210 is the intelligent core and scheduling center of the entire system. Its functionality has been enhanced, possessing not only the read / write management capabilities of a standard memory controller 210, but also the special functions proposed in this invention. Specifically, the memory controller 210 is responsible for writing matrix data and input vector data into multiple memory banks of the DDR memory 220 according to a preset address mapping strategy; generating and sending a precisely orchestrated sequence of computational commands to the DDR memory 220 to schedule these memory banks to perform cascaded computations; and, after the computation is completed, retrieving the final computation result from the last-level memory bank.

[0099] The DDR memory 220 is the execution unit for computation. This DDR memory 220 is a standard, commercially available DDR4 or DDR5 memory module, requiring no physical modifications. Its multiple memory modules, upon receiving a special sequence of computational commands from the memory controller 210, can be scheduled to work collaboratively, performing internal data transfer and accumulation calculations between memory modules.

[0100] This system, through the collaborative operation of the memory controller 210 and standard DDR memory 220, constructs a high-efficiency, low-power in-memory computing system that requires no special hardware. The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A matrix-vector multiplication implementation method based on DDR4 / 5Bank, characterized in that, include: Matrix data and input vector data are written into multiple storage banks that are cascaded in sequence in DDR memory, wherein the DDR memory is standard DDR4 memory or DDR5 memory; Generate and send a sequence of computation commands to the DDR memory to schedule the multiple memory banks to perform cascaded computations; The calculation command sequence includes: A first command is sent to the storage group that serves as the previous stage in the cascaded computation, so that it generates the computation result of the previous stage and sends the computation result of the previous stage to the storage group that serves as the next stage. After a preset column operation delay time, a second command is sent to the next-level storage group to generate the next-level calculation result and add it to the received previous-level calculation result; The final calculation result is obtained from the storage group that is the last stage in the cascaded calculation.

2. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 1, characterized in that, The step of writing matrix data and input vector data into multiple cascaded memory banks in DDR memory includes: The memory controller writes the matrix data sub-blocks to be calculated and the input vector data into multiple storage banks that are cascaded in sequence in the DDR memory, according to a preset address mapping strategy.

3. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 2, characterized in that, The preset address mapping strategy includes: decomposing the matrix data sub-blocks into multiple consecutive data sub-blocks according to a preset decomposition logic, and sequentially mapping the multiple consecutive data sub-blocks and their corresponding input vector data to multiple memory banks that are sequentially cascaded in the DDR memory.

4. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 1, characterized in that, The computation command sequence further includes sending an activation command to the plurality of memory groups before sending the first command, so that the corresponding row buffer is in a ready state.

5. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 1, characterized in that, The step of sending a first command to the memory bank group serving as the preceding stage in the cascaded computation to generate the computation result of the preceding stage includes: Based on the received first command, the weight matrix data and input vector data are read from the storage array of the previous level storage group, and the previous level calculation result is generated by performing simulation calculation using peripheral circuits or multiplication operation through digital logic.

6. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 1, characterized in that, The step of sending the calculation result of the previous stage to the storage group that serves as the next stage includes: The calculation results of the previous stage are passed to the memory group of the next stage through a cascading path.

7. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 1, characterized in that, During the execution of the cascaded computation, the method further includes: Perform diagnostic operations periodically and measure the actual physical execution time of the diagnostic operations; Based on the deviation between the actual physical execution time and the preset theoretical execution time, the target count value of the clock cycle counter corresponding to the column operation delay time is dynamically adjusted to ensure that the column operation delay time meets the predefined inter-memory block column operation interval.

8. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 7, characterized in that, The diagnostic operations include a combination of read and write commands sent to the same or different memory groups with a fixed execution time.

9. The matrix-vector multiplication implementation method based on DDR4 / 5Bank according to claim 1, characterized in that, After obtaining the final calculation result, the following is included: The matrix-vector multiplication calculation is re-executed using the memory controller based on the original weight matrix and input vector data to obtain the verification calculation results; The verification calculation result is compared with the final calculation result; If the comparison results are inconsistent, the calculation command sequence is resent or the weight matrix and input vector data are written into a preset set of backup storage units and the preset set of backup storage units is scheduled to perform cascaded calculations.

10. A matrix-vector multiplication implementation system based on DDR4 / 5Bank, characterized in that, The system includes: A memory controller is configured to write matrix data and input vector data into multiple cascaded storage banks of DDR memory, wherein the DDR memory is standard DDR4 memory or DDR5 memory; it is also configured to generate and send a sequence of computation commands to the DDR memory to schedule the multiple storage banks to perform cascaded computation; and it is also configured to obtain the final computation result from the storage bank that is the last stage in the cascaded computation. DDR memory is used to receive computation command sequences and schedule the multiple memory blocks to perform cascaded computations; The calculation command sequence includes: A first command is sent to the storage group that serves as the previous stage in the cascaded computation, so that it generates the computation result of the previous stage and sends it to the storage group that serves as the next stage. After a preset column operation delay time, a second command is sent to the next-level storage group to generate the next-level calculation result and add it to the received previous-level calculation result.

Images