A new coarse-grained configurable architecture based on in-memory computing technology

Abstract

The present invention discloses a novel coarse-grained configurable architecture based on in-memory computing technology, involving SRAM technology. This solution is proposed to address four major issues in the prior art, such as reliance on on-chip cache. It includes a dispersion / aggregation module, a management module, a conversion module, and an in-memory computing macro. The in-memory computing macro is used to perform calculations under the control of the management module. Its advantages are that, compared with other coarse-grained configurable architecture technology solutions based on von Neumann architecture processing engines, it has the characteristics of smaller area, lower power consumption, and higher throughput. At the same time, a feasible solution is proposed to the four major challenges of SRAM in-memory computing technology: in terms of utilization, the advantages of in-memory computing technology in computing power density and energy efficiency methods are fully utilized; the computing architecture method flexibly determines the on-chip cache size of the coarse-grained configurable architecture; in terms of computing completeness, it meets the computing requirements of more applications; in terms of software ecology, it allows it to be called by RISC‑V CPU and supports compilation and acceleration of applications under the RISC‑V ecology.

Description

Technical Field

[0001] The present invention relates to SRAM technology, and in particular to a novel coarse-grained configurable architecture based on in-memory computing technology. Background Art

[0002] While SRAM in-memory computing technology has become one of the mainstream integrated storage and computing solutions due to its high process maturity and strong robustness, it still faces key issues and challenges, including unit circuit design, reliability, utilization, computing architecture, computational completeness, and software ecosystem. Regarding utilization, due to the diverse types and sizes of operators, deploying different real-world applications on a fixed-size, fixed-operator in-memory computing array can significantly reduce computing resource utilization, hindering the full utilization of SRAM in-memory computing technology in terms of computing power density and energy efficiency. Regarding computing architecture, since some data, such as intermediate calculation results, still require on-chip cache during algorithm execution, the limited on-chip cache capacity may become a new system bottleneck, significantly increasing off-chip memory access overhead. Regarding computational completeness, although in-memory computing can significantly improve computing power density and energy efficiency for matrix-matrix operations, a complete application in real-world scenarios still requires scalar, vector, and nonlinear operations, which places new demands on the computational completeness of SRAM-based in-memory computing accelerators. Regarding the software ecosystem, in addition to optimizing and improving the underlying circuitry, implementing in-memory computing technology also requires a supporting compiler and toolchain. Designing a compatible instruction set that integrates with the existing ecosystem to facilitate compilation and application acceleration is a critical consideration. Summary of the Invention

[0003] The purpose of the present invention is to provide a novel coarse-grained configurable architecture based on in-memory computing technology to solve the problems existing in the above-mentioned prior art.

[0004] The present invention provides a novel coarse-grained configurable architecture based on in-memory computing technology, comprising:

[0005] The scatter / gather module is used to split or pack data of different precisions, interact with external modules, and organize sparse data into dense data and send it to the management module;

[0006] The management module is used to decode the instructions and data transmitted by the host, convert the instructions into control commands of the in-memory calculation macro, and send the control commands and data to the in-memory calculation macro at the corresponding address to participate in the calculation;

[0007] A conversion module, used to transmit signals to a specified address to build an on-chip network;

[0008] The in-memory computation macro includes a 512x256 8T SRAM array, a computational logic unit supporting addition, subtraction, and partial products, a set of addition trees, a control unit, a timing control circuit, and peripheral circuits. The in-memory computation macro is used to perform operations under the control of the management module.

[0009] The scatter / gather module includes two buffers, a left shifter, and a right shifter; external data is divided into a header and input data;

[0010] The header is used to determine whether the input data is an instruction or a data signal involved in calculation. If the input data is an instruction, it is directly sent to the management module; otherwise, it is sent to the cache for splicing.

[0011] The management module includes two data interaction modules, two CIM managers, a scratchpad memory SPM, a CIM mapping register, and a crossbar switch matrix; the instructions and data signals transmitted from the scatter / gather module are respectively sent to the CIM mapping register and the data interaction module;

[0012] The CIM mapping register is used to store the data movement path and the function selection and mode selection of each in-memory calculation macro. The CIM mapping register outputs commands to the CIM manager;

[0013] The CIM manager is used to manage the conversion modules and in-memory calculation macros of the row;

[0014] The data interaction module is used to receive the data involved in the calculation and send it to the bank for calculation or to the adjacent management module based on the judgment conditions.

[0015] The conversion module includes four data interaction modules for receiving and transmitting data to adjacent conversion modules or in-memory computing macros.

[0016] The calculation process of the in-memory calculation macro is as follows:

[0017] The first step is to select whether to enable the multi-function support computing logic unit MFU and addition tree module according to the operator type;

[0018] The second step is to configure the function selection of the multi-function support computing logic unit MFU;

[0019] The third step is to write data into the SRAM array;

[0020] Step 4: Read two sets of data at the same time for calculation and output the results to the conversion module S;

[0021] The fifth step is to determine whether the calculation is completed. If it is completed, the calculation is terminated and the next round of calculation is awaited. If it is not completed, the data is continued to be read and the calculation is continued until the calculation is completed.

[0022] The novel coarse-grained configurable architecture based on in-memory computing technology described in the present invention has the advantages of smaller area, lower power consumption, and higher throughput compared to other coarse-grained configurable architecture technology solutions based on von Neumann architecture processing engines. At the same time, a feasible solution is proposed for the four major challenges of SRAM in-memory computing technology: Regarding utilization, the in-memory computing macro supports five computing methods: vector addition, vector subtraction, vector-matrix multiplication, matrix-matrix multiplication, and comparison operation, fully leveraging the advantages of in-memory computing technology in computing power density and energy efficiency. Regarding computing architecture methods, the in-memory computing mode and memory mode of the in-memory computing macro can flexibly determine the on-chip cache size of the coarse-grained configurable architecture, alleviating the system bottleneck caused by limited on-chip cache. Regarding computing completeness, the in-memory computing macro supports not only matrix operations but also vector operations, meeting the computing requirements of more applications. Regarding software ecology, the proposed architecture allows it to be called by the RISC-V CPU, supporting the compilation and acceleration of applications in the RISC-V ecosystem. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a structural diagram of the novel coarse-grained configurable architecture described in the present invention.

[0024] Figure 2 It is a structural diagram of the dispersion / aggregation module in the present invention.

[0025] Figure 3 It is a structural diagram of the management module in the present invention.

[0026] Figure 4 It is a structural diagram of the conversion module in the present invention.

[0027] Figure 5 It is a structural diagram of the data flow control unit in the present invention.

[0028] Figure 6 It is a structural diagram of the in-memory calculation macro described in the present invention.

[0029] Figure 7 It is a schematic diagram of the method of the in-memory calculation macro in the present invention to simultaneously read out the three bit lines WBL, WBLb, and RBL.

[0030] Figure 8 It is a structural diagram of the multifunctional support computing logic unit described in the present invention.

[0031] Figure 9 It is a schematic diagram of the calculation process of the novel coarse-grained configurable architecture described in the present invention.

[0032] Figure 10It is the instruction representation of the RISC-V extended instruction set matched with the new coarse-grained configurable architecture described in the present invention.

[0033] Figure 11 It is a schematic diagram of the calculation flow of the in-memory calculation macro described in the present invention.

[0034] Figure 12 It is a timing diagram of the calculation process of the in-memory calculation macro described in the present invention.

[0035] Figure 13 It is a schematic diagram of the addition calculation process of the calculation logic unit described in the present invention.

[0036] Figure 14 It is a schematic diagram of the subtraction calculation process of the calculation logic unit described in the present invention.

[0037] Figure 15 It is a schematic diagram of the partial product calculation process of the calculation logic unit described in the present invention.

[0038] Figure 16 This is a schematic diagram of the interactive communication between the novel coarse-grained configurable architecture described in the present invention and the RISC V CPU processor.

[0039] Reference numerals:

[0040] Scatter / gather module S / G Unit, management module MU, conversion module S, in-memory computing macro CIM;

[0041] Precharge module Precharge, read word line decoder RWL Decoder, write word line decoder WWL Decoder, bit line driver BL Driver, sense amplifier SA, multi-function support calculation logic unit MFU, adder tree Adder Tree, control circuit Control Unit, timing control circuit Timing. DETAILED DESCRIPTION

[0042] The present invention describes a novel coarse-grained configurable architecture SHWCIM based on in-memory computing technology, such as Figure 1The module includes a scatter / gather module, which splits or packages data of different precisions, such as INT 8, INT 16, INT 32, and INT 64, to interact with external modules. It also organizes sparse data into dense data for transmission to the management module, improving data processing efficiency. The management module decodes instructions and data transmitted by the host, converts them into control commands for in-memory computation macros, and sends the control commands and data to the in-memory computation macros at the corresponding addresses for computation. The conversion module, acting as a relay station for various signals, transmits them to the modules at the specified addresses, creating a network-on-chip (NOC). The multi-functional, configurable in-memory computation macros include a 512x256 8T SRAM array, a computational logic unit supporting addition, subtraction, and partial products, a set of adder trees, a control unit, and other peripheral circuits.

[0043] Scatter / gather module S / G Unit Figure 2 As shown, it includes two buffers, a left shifter (LeftShifter), and a right shifter (RightShifter). External data is divided into a header flag and input data. When the header is FF, it indicates that the input data is an instruction and is directly sent to the management module MU. When the header is data other than FF, it indicates that the input data is a data signal involved in calculations and is sent to the buffer for splicing. Regardless of the data precision (8b / 16b / 32b / 64b), it is spliced ​​into a 256b data bit width and sent to the management module MU, completing the data packing process. When the SHWCIM data is output, data unpacking is required to unpack the 256b data in the opposite manner to the data with the required precision of the system.

[0044] The management module MU is as follows Figure 3 As shown, it includes two data interaction modules, two CIM managers, a scratchpad memory SPM, a CIM mapping register, and a crossbar switch matrix Crossbar. The instructions and data signals transmitted from the scatter / gather module S / G Unit are respectively sent to the CIM mapping register and the data interaction module. The CIM mapping register is used to store the data movement path in the SHWCIM, that is, the routing table, as well as the function selection and mode selection of each multi-function support configurable in-memory calculation macro. The CIM mapping register outputs the command CMD to the CIM manager. The CIM manager is used to manage the conversion module and the multi-function support configurable in-memory calculation macro of the row. Each row unit has a CIM manager that is only used for the units in this row. The data interaction module is used to receive the data participating in the calculation and send it to the row for calculation or to the adjacent management module MU according to the judgment conditions.

[0045] The conversion module is as follows Figure 4 As shown, it contains four data exchange modules for receiving and transmitting data to adjacent conversion modules or multi-function support configurable in-memory calculation macros. The data exchange module contains a FIFO to unify the timing and reduce the bit error rate. It also contains a data flow control unit, the specific structure is as follows Figure 5 The data flow control unit receives data from the four data interaction modules and instructs the arbiter through the control register to guide the input data of channel 0, channel 1, channel 2, and channel 3 to the output data of the specified channel, and then passes it to the corresponding data interaction module.

[0046] In-memory computation macros such as Figure 6 As shown, it includes a 512x 256 8T SRAM storage array, a multi-function support calculation logic unit MFU, a set of addition trees, a control logic circuit, a timing control circuit and other peripheral circuits. The 8T SRAM has two separate word lines WWL and RWL, and three separate bit lines WBL, WBLb and RBL, allowing two sets of data to be read out simultaneously. The specific method is as follows Figure 7 When matrix multiplication is required, the adder tree circuit receives the partial product calculation results and, through a series of addition operations, produces the matrix multiplication result. Furthermore, the control logic circuit and timing control circuit are used to receive control signals and precisely control the timing of reads, writes, and calculations, respectively, to improve computational efficiency. When different rows of WWL and RWL are selected, the 512x 256 8T SRAM memory array can simultaneously output up to two sets of 256-bit data. This data is then sent to the multi-function support computation logic unit (MFU) via the sense amplifier SA for computation.

[0047] The multi-function support calculation logic unit MFU supports two groups of 256-bit data for addition, subtraction and partial product operations. The specific methods are as follows Figure 8 This unit supports vector addition, vector subtraction and matrix multiplication operations, and the subtraction carry input C in_sub ,WBLb [i] , WBL [i] , RBL [i] , addition carry input Cin_add, subtraction carry output C out_sub , Addition carry output C out_add , Sum / Difference of addition / subtraction, Partial Product of partial product, the specific operation process is as follows Figures 13 to 15 shown.

[0048] A typical calculation process is as follows Figure 9As shown in the figure: First, the host issues instructions based on the RISC-V extended instruction set to configure the data movement path, or routing method, of the SHWCIM. Second, the host issues instructions based on the RISC-V extended instruction set to configure the function selection of each in-memory computation macro in the SHWCIM, including vector addition, vector subtraction, vector-matrix multiplication, matrix-matrix multiplication, and comparison operations, as well as selecting whether the current in-memory computation macro is in in-memory computation mode or memory mode. Third, the data is stored in off-chip memory. The host issues a DMA call instruction to efficiently write the data from the off-chip memory to the SHWCIM via DMA. Using the previously configured routing method, the data is written to the corresponding in-memory computation macro for computation, and the computation results are stored in a nearby in-memory computation macro in memory mode. Fourth, the host issues a read instruction based on the RISC-V extended instruction set to read the computation results from the SHWCIM and store them in off-chip DRAM. This completes a typical SHWCIM computation flow.

[0049] Figure 10 The instruction table shown contains 9 RISC-V extended instructions with a 32-bit instruction width. From top to bottom, they are data precision configuration instructions, data memory / in-memory calculation macro quantity configuration instructions, SHWCIM routing table configuration instructions, vector addition operation instructions, vector subtraction operation instructions, vector-matrix multiplication operation instructions, matrix-matrix multiplication operation instructions, data loading instructions, and data storage instructions.

[0050] The calculation method of the in-memory calculation macro is as follows Figure 11 As shown. First, based on the operator type, select whether to enable the multi-function support calculation logic unit MFU and the addition tree module. Second, configure the function selection of the multi-function support calculation logic unit MFU. Third, write the data to the SRAM array. Fourth, read two sets of data simultaneously for calculation and output the results to the conversion module S. Fifth, determine whether the calculation is complete. If it is, end the calculation and wait for the next round of calculation. If it is not, continue reading data for calculation until the calculation is complete.

[0051] The calculation process sequence of the in-memory calculation macro is as follows Figure 12As shown in the figure, this is an example of the matrix multiplication calculation process. First, the multi-function support calculation logic unit enable MFU_ENABLE and the adder tree module enable Adder_Tree_ENABLE send a pulse signal to turn on the calculation function. Secondly, the multi-function support calculation logic unit function selection signal MFU_CONFIG is configured to 1001, indicating that this calculation is a matrix-matrix multiplication calculation. Third, the storage array write enable CIM_WR_ENABLE is turned on and the CIM_WR_DATA data is written. Fourth, CIM_WR_ENABLE is turned off and the storage array read enable CIM_RD_ENABLE is turned on to start the calculation.

[0052] The addition operation process of the calculation logic unit is as follows: Figure 13 As shown, the two bits of data read out by WBL and RBL are input, and the carry bit of the calculation logic unit supported by the low-order multi-function is input, and the calculation result is output and the carry bit is output to the higher-order calculation logic unit. The subtraction operation process of the calculation logic unit is as follows Figure 14 As shown, using the characteristic that WBLb is the inverse code of WBL, the inverse code is added by 1 to obtain the complement code, and then the addition operation is performed on the data read out by RBL to obtain the subtraction operation result. The partial product operation process of the calculation logic unit is as follows: Figure 15 As shown, the data read out from WBL and RBL are subjected to a logical AND operation to obtain a partial product result, which is then sent to the addition tree for operation to obtain the final result.

[0053] The interaction between the novel coarse-grained configurable architecture and the RISC V CPU processor described in the present invention is as follows Figure 16 As shown in the figure, SHWCIM directly obtains instructions from the instruction decoding module of the RISC-V CPU, and the SHWCIM bus is set separately from the RISC-V CPU instruction bus and data bus to improve computing efficiency.

[0054] Those skilled in the art can make various other corresponding changes and deformations based on the technical solutions and concepts described above, and all of these changes and deformations should fall within the scope of protection of the claims of the present invention.

Claims

1. A new coarse-grained configurable architecture based on in-memory computing technology, characterized by: include: The scatter / gather module is used to split or pack data of different precisions, interact with external modules, and organize sparse data into dense data and send it to the management module; The management module is used to decode the instructions and data transmitted by the host, convert the instructions into control commands of the in-memory calculation macro, and send the control commands and data to the in-memory calculation macro at the corresponding address to participate in the calculation; A conversion module, used to transmit signals to a specified address to build an on-chip network; In-memory computation macro, consisting of a 512x256 8T SRAM array, a computational logic unit supporting addition, subtraction, and partial products, a set of addition trees, a control unit, a timing control circuit, and peripheral circuits; The in-memory calculation macro is multifunctional and configurable, and is used to perform calculations under the control of the management module and perform function selection and mode selection according to the control command; the in-memory calculation macro also includes a memory mode.

2. The novel coarse-grained configurable architecture based on in-memory computing technology according to claim 1, characterized in that: The scatter / gather module includes two buffers, a left shifter, and a right shifter; external data is divided into a header and input data; The header is used to determine whether the input data is an instruction or a data signal involved in calculation. If the input data is an instruction, it is directly sent to the management module; otherwise, it is sent to the cache for splicing.

3. The novel coarse-grained configurable architecture based on in-memory computing technology according to claim 1, characterized in that: The management module includes two data interaction modules, two CIM managers, a scratchpad memory SPM, a CIM mapping register, and a crossbar switch matrix; the instructions and data signals transmitted from the scatter / gather module are respectively sent to the CIM mapping register and the data interaction module; The CIM mapping register is used to store the data movement path and the function selection and mode selection of each in-memory calculation macro. The CIM mapping register outputs commands to the CIM manager; The CIM manager is used to manage the conversion modules and in-memory calculation macros of the row; The data interaction module is used to receive the data involved in the calculation and send it to the bank for calculation or to the adjacent management module based on the judgment conditions.

4. The novel coarse-grained configurable architecture based on in-memory computing technology according to claim 1, characterized in that: The conversion module includes four data interaction modules for receiving and transmitting data to adjacent conversion modules or in-memory computing macros.

5. The novel coarse-grained configurable architecture based on in-memory computing technology according to claim 1, characterized in that: The calculation process of the in-memory calculation macro is as follows: The first step is to select whether to enable the multi-function support computing logic unit MFU and addition tree module according to the operator type; The second step is to configure the function selection of the multi-function support computing logic unit MFU; The third step is to write data into the SRAM array; Step 4: Read two sets of data at the same time for calculation and output the results to the conversion module S; The fifth step is to determine whether the calculation is completed. If it is completed, the calculation is terminated and the next round of calculation is awaited. If it is not completed, the data is continued to be read and the calculation is continued until the calculation is completed.

Images