A real-time multi-port parallel read-write near-memory processor

Abstract

The application belongs to the technical field of integrated circuit design, and relates to a real-time multi-port parallel read-write near-memory processor. The near-memory processor adopts a multi-core shared memory near-memory design, a self-defined atomic-level instruction architecture and a customized real-time variable-precision multiplier-accumulator, and comprises a plurality of near-memory computing cores connected with a shared memory. The near-memory computing cores further have an instruction storage space, and control port inputs atomic-level instructions. Each near-memory computing core further comprises a real-time variable-precision multiplier-accumulator, and the multiplier-accumulator adopts an iterative structure. The processor can refresh data in a data buffer at any time and read and write data in the shared memory in real time. Various rich operators are composed by controlling the computing module to support neural network operations other than multiplier-accumulator calculations. Partial product results of different bit widths are combined through an adder, which can adapt to different bit widths and complete a variety of precision vector inner products. The multiplier-accumulator can still complete multiplier-accumulator operations within a single clock cycle at a high main frequency, thereby providing extremely high real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of integrated circuit design technology and relates to a real-time multi-port parallel read / write near-memory processor. Background Technology

[0002] In recent years, in-memory computing (CIM) technology has emerged, aiming to overcome the von Neumann bottleneck, reduce memory access in artificial intelligence (AI) tasks, and thus improve energy efficiency for edge computing tasks requiring low-power machine learning. However, traditional CIM models lack flexibility and programmability, and still suffer from redundant memory accesses, which reduce overall energy efficiency and real-time performance. First, traditional CIM architectures assume that weights are not updated or are updated very rarely in neural network (NN) operations. However, newer models such as the transformer have dynamic weights when calculating self-attention, and some weights are updated dynamically in real time with the input, thus requiring continuous refresh of storage content. Second, although CIM optimizes multiply-accumulate operations, for some critical complex non-multiply-accumulate operations not supported by the CIM core, data must be transported to an off-chip processor for computation before the results are written back, creating a performance bottleneck due to inter-chip throughput. Finally, the precision of each layer in a neural network (NN) contributes differently to the accuracy. Low-precision quantization can be used for layers with low contribution and high-precision quantization can be used for layers with high contribution, so that there is almost no loss of precision, while greatly reducing inference time and storage load. However, in traditional CIM calculation, fixed quantization precision is often used or complex configuration is required to support variable precision operation.

[0003] Therefore, improving the parallelism and real-time performance of near-memory processors is an urgent problem to be solved, and it is also the problem that this application aims to solve. Summary of the Invention

[0004] The purpose of this invention is to overcome the problems of poor flexibility, low programmability, and a large number of redundant memory accesses in existing in-memory computing models, and to further improve the overall circuit energy efficiency and real-time performance of the operation. A real-time multi-port parallel read-write near-memory processor is proposed. The near-memory processor adopts a multi-core shared memory near-memory design, a custom atomic-level instruction architecture, and a custom real-time variable precision multiply-accumulate unit. It can complete vector inner products of various precisions such as 4-bit / 8-bit / 16-bit without configuration. Even at high clock frequencies, it can still complete multiply-accumulate operations in a single cycle, thus providing extremely high real-time performance.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A real-time multi-port parallel read / write near-memory processor includes a data port and a control port, wherein the data port and the control port are connected to a near-memory processor module, characterized in that it further includes a near-memory processor module and a shared memory;

[0007] The near memory processor module includes multiple near memory computing cores, each connected to the shared memory, and can refresh the data in the data cache and read and write data in the shared memory in real time.

[0008] Each near-memory computing core includes a computing module, which in turn includes a multiply-accumulator. The multiply-accumulator includes a real-time variable-precision multiply-accumulator with an iterative structure. The multiply-accumulator combines the partial product results of different bit widths through an adder, which can adapt to the calculation of the inner product of vectors with different bit widths.

[0009] The control port inputs atomic-level instructions and controls the computing module to form a variety of rich operators to support neural network operations other than multiplication and addition.

[0010] The shared memory includes a multi-port unit, a memory unit, and unit logic control; the near-memory computing core also includes a data cache, an instruction storage space, and a memory array read / write controller.

[0011] The data buffer caches data for the computing module and the memory array read / write controller to read; the data buffer is connected to the data port;

[0012] The data port is the location for data to enter and exit, and each near-memory computing core inputs or outputs several bits of data each time.

[0013] The instruction storage space includes an instruction buffer, an instruction fetching module, a value retrieval logic module, and an instruction decoder; the instruction decoder controls the data buffer, the computing module, and the memory array read / write controller via a connection; the control port is connected to the instruction storage space of the near-memory computing core.

[0014] The connection is a one-way connection initiated by the instruction decoder in the instruction storage space;

[0015] The computing module also includes a divider, a comparator, an adder, and a bit manipulator; the computing module exchanges data with the data buffer and the memory array read / write controller; the data buffer and the memory array read / write controller exchange data through a bidirectional connection; the memory array read / write controller is connected to the shared memory and exchanges data with it.

[0016] The computing module performs various operations on the input data and outputs the results to the data buffer and the memory array read / write controller; the input data refers to the data flowing into the instruction decoder, data buffer, and memory array read / write controller connected to the computing module.

[0017] The memory array read / write controller controls the flow of data, controls the position of the write unit in the shared memory, and controls the direction of data flow when it is read out.

[0018] The shared memory includes several memory units, and each memory unit has multiple multi-port units;

[0019] The function of the shared memory is to store data for the near-memory processor to read and write;

[0020] The multi-port unit includes a unit logic control module and several unit modules.

[0021] The near-memory processor includes read mode and write mode;

[0022] The read mode involves reading data stored in the shared memory and processing it in the near-memory processor module. The input signal enters the instruction storage space through the control port, and then controls the memory array read / write controller. Based on the location information of the multi-port unit in the memory cell to be read contained in the input signal, the data is transmitted to the shared memory. After receiving the location information to be read, the shared memory controls the corresponding memory cell to respond, and the cell logic controls the output of data at that location. The output data is stored in the data buffer or directly output from the data port. When the control port loads the instruction set compiled by the assembler into the instruction storage space of the arithmetic core, the operation of the arithmetic core is controlled through instruction decoding. When the calculation module is called to perform the calculation, the output information enters the calculation module to perform the corresponding instruction calculation. After the calculation is completed, the data may enter the data buffer or be directly output from the port.

[0023] In the write mode, the input signal enters the data buffer from the data port or the data itself is stored in the data buffer; then the data in the data buffer enters the memory array read / write controller and is written to the corresponding shared memory unit; the input signal can also be loaded into the instruction storage space of the arithmetic core by loading the instruction set compiled by the assembler, and the operation of the arithmetic core is controlled by the instruction decoding, calling the calculation module to perform the operation, and then passing it to the data buffer to control the read / write array to write the corresponding data to the shared memory.

[0024] The location information of the multi-port unit in the memory cell to be read is L-bit location information.

[0025] The plurality of near-memory computing cores are N, denoted as the first near-memory computing core to the Nth near-memory computing core; and N is greater than or equal to 4 and less than or equal to 16; each near-memory computing core inputs or outputs several bits of data in the form of L bits, and the bit width of the near-memory computing core is L bits; where L is greater than or equal to 4 and less than or equal to 64.

[0026] The data port can input or output a maximum of L*N bits of data.

[0027] The number of memory units is Z; Z is greater than or equal to 4 and less than or equal to 8; the number of multi-port units is M; the value of M is greater than 1k and less than 4k.

[0028] The number of the plurality of unit modules is L, which is the same as the location information of the multi-port unit in the memory unit and the bit width of the near-memory computing core; denoted as CELL0 to CELL1. L-1 Each multi-port unit controls one L-bit of data; each near-memory computing core is connected to all memory units for reading and writing L-bit data.

[0029] Beneficial effects

[0030] This invention provides a real-time multi-port parallel read / write near-memory processor, which, compared with other existing CIM (Continuous In-Memory) architecture memory, has the following advantages:

[0031] 1. The near-memory processor adopts a multi-core shared memory near-memory computing design. Multiple near-memory operation cores can refresh their own memory area or read and write the shared memory area in real time, thereby ensuring that the total memory access is reduced when the operation involves dynamic weight updates and input propagation.

[0032] 2. The near-memory processor adopts a custom atomic-level ISA to support basic operations (addition, subtraction, multiplication, and division), conditional operations, comparison operations, and bit operations. This allows for the formation of a variety of rich operators within the near-memory computing core through these atomic instructions to support neural network operations other than multiplication and addition.

[0033] 3. The near-memory processor adopts a custom real-time variable precision multiply-accumulator, which can complete vector inner products in various precision conditions such as 4-bit / 8-bit / 16-bit without configuration, thus providing good support for the inference task of variable precision neural network models and still being able to complete multiply-accumulator operations within a single clock cycle at high clock frequency, thereby providing high real-time performance. Attached Figure Description

[0034] Figure 1 This invention relates to the overall architecture of a real-time multi-port parallel read / write near-memory processor.

[0035] Figure 2 This is a diagram of a near-memory computing scheme for a real-time multi-port parallel read / write near-memory processor according to the present invention.

[0036] Figure 3 This is a flowchart of the self attention calculation process in the transformer during the specific implementation of the real-time multi-port parallel read / write near-memory processor of the present invention.

[0037] Figure 4 This is a diagram illustrating the implementation process of the self attention part in the transformer during a specific implementation of the real-time multi-port parallel read / write near-memory processor of this invention. Detailed Implementation

[0038] The following description, in conjunction with the accompanying drawings and embodiments, further illustrates a specific implementation of the real-time multi-port parallel read / write near-memory processor of the present invention.

[0039] Example 1

[0040] This invention relates to a real-time multi-port parallel read / write near-memory processor. The near-memory processor employs a multi-core shared memory near-memory design, a custom atomic-level instruction architecture, and a custom real-time variable-precision multiply-accumulate unit. It includes multiple near-memory computing cores connected to the shared memory. Each near-memory computing core contains instruction storage space, and atomic-level instructions are input through control ports. Each near-memory computing core's computing module includes a real-time variable-precision multiply-accumulate unit, which uses an iterative structure. The processor can refresh data in the data cache and read / write data in the shared memory in real time. By controlling the computing modules to form various rich operators, it supports neural network operations other than multiply-accumulate operations. By combining partial product results of different bit widths through adders, it can adapt to different bit widths and complete vector inner products of various precisions. Even at high clock frequencies, it can still complete multiply-accumulate operations within a single clock cycle, thus providing extremely high real-time performance.

[0041] This embodiment details the connection and signal flow of a real-time multi-port parallel read / write near-memory processor under normal operating conditions.

[0042] The near memory processor includes a near memory processor module and a shared memory, and the near memory processor module and the shared memory are connected together.

[0043] The near memory processor module includes N=4 near memory computing cores, namely near memory computing core 1 to near memory computing core 4;

[0044] The near-memory computing cores 1 to 4 each include a data cache, an instruction storage space, a computing module, and a memory array read / write controller;

[0045] The data buffer is connected to the data port;

[0046] The data port is the location for data input and output. Each near-memory computing core can input or output one L=16 bits of data at a time, that is, the data port can input or output a maximum of 16*4 bits of data.

[0047] The instruction storage space includes an instruction buffer, an instruction fetching module, a value retrieval logic module, and an instruction decoder; the instruction decoder is connected to the control data buffer, the calculation module, and the memory array read / write controller, and all connections are unidirectional connections originating from the instruction storage space;

[0048] The computing module includes a divider, a real-time variable precision multiply-accumulator, a comparator, an adder, and a bit manipulator; the computing module exchanges data with the data buffer and the memory array read / write controller; the data buffer and the memory array read / write controller also have a bidirectional data exchange connection; the memory array read / write controller is connected to the shared memory and shares data.

[0049] The shared memory comprises Z = 6 memory units; each memory unit includes 1k multi-port units.

[0050] The multi-port unit includes a unit logic control and L=16 unit modules, namely CELL0 to CELL16. 15 Each multiport unit controls one 16-bit data.

[0051] Each near-memory computing core is connected to all memory units and can read and write 16 bits of data.

[0052] The signal flow relationships among the modules in the near-memory processor are as follows:

[0053] The near-memory processor is divided into two types of operations: reading and writing.

[0054] During read operations, data stored in shared memory is read and processed by the nearest memory calculator. The input signal contains the location information of the cell to be read. It enters the instruction storage space through the control port and then controls the memory array read / write controller. The location information of a specific 16-bit data to be read is transmitted to the shared memory. After receiving the location information, the corresponding memory cell responds, and the cell logic controls the output of the 16-bit data at that location. The output information can be stored in the data buffer or directly output from the data port. When the control port loads the custom ISA compiled by the assembler into the instruction storage space of the arithmetic core, it controls the operation of the arithmetic core through instruction decoding. When the calculation module is called to perform the operation, the output information enters the calculation module to perform the corresponding instruction operation. After the operation is completed, it may enter the data buffer or be directly output from the port.

[0055] During write operations, input signals enter the data buffer from the data port, or the data itself is stored in the data buffer. Then, the data in the data buffer enters the memory array read / write controller and is written to the corresponding shared memory unit. Input signals can also be loaded into the instruction storage space of the arithmetic core by loading a custom ISA compiled by the assembler, and the operation of the arithmetic core is controlled by instruction decoding. The arithmetic module is called to perform the operation, and then the data is passed to the data buffer to control the read / write array to write the corresponding data to the shared memory.

[0056] Each of the four processing cores has a read / write port to the shared memory and can operate independently, enabling simultaneous four-read and four-write operations. This effectively quadruples the array's speed and capacity while reducing area overhead to only three times that of a single-channel read / write operation, and this model offers excellent scalability. During multiplication-accumulation operations, the processing cores decode the MAC instruction, read the memory unit, and send it to the computation module to complete the multiplication and accumulation. This process is completed within 2000ps, thus eliminating the need for pipelining. At a 500MHz clock frequency, the MAC operation can be completed within a single clock cycle, providing high real-time performance. By rationally arranging and allocating atomic-level instructions, various nonlinear operations can be applied.

[0057] Example 2

[0058] This embodiment details the input and output behavior of a multi-port parallel read / write high real-time near-memory processor under normal operating conditions.

[0059] Taking self-attention in transformers as an example,

[0060] The operation of the multi-port high real-time near-memory processor includes the following steps:

[0061] Step 1: Write the static weight input X to the common storage area and read the weight into the computing core Core1-4;

[0062] Step 2: After the kernel accesses the common memory area to obtain X, calculate K, Q, and V;

[0063] Step 3: The results of Q and V are directly stored in Core 2 and 3 as weights. The result of K is written back to the common storage area and then read out into the core that stores Q as input for calculation.

[0064] Step 4: Obtain K T After performing a non-linear operation on *Q, A is obtained, and finally shared with Core3 through the public storage area for V*A calculation.

[0065] Figure 1The system's overall architecture is demonstrated, including: a near-memory processor and shared memory; the near-memory processor module includes four near-memory computing cores, namely near-memory computing core 1 to near-memory computing core 4; each of near-memory computing cores 1 to 4 includes a data buffer, instruction storage space, computing module, and memory array read / write controller; the data buffer is connected to a data port; the data port is the data input / output location, and each near-memory computing core can input or output one 16-bit data at a time, meaning the data port can input or output a maximum of 16*4 bits of data; the instruction storage space includes an instruction buffer, instruction fetch module, value retrieval logic module, and instruction decoder; the instruction decoder is connected to the control data buffer. The registers, computing modules, and memory array read / write controller are all connected via unidirectional connections originating from the instruction storage space. The computing module includes a divider, a real-time variable-precision multiplier-accumulator, a comparator, an adder, and bit manipulators. Data exchange occurs between the computing module and the data buffer and the memory array read / write controller. A bidirectional data exchange connection also exists between the data buffer and the memory array read / write controller. The memory array read / write controller is connected to the shared memory, enabling data sharing. The shared memory comprises six memory cells. Each memory cell includes 1k multi-port units. Each multi-port unit includes one unit logic control and 16 unit modules, namely CELL0 to CELL16. 15 Each multi-port unit controls one 16-bit data; each near-memory computing core is connected to all memory units and can read and write 16-bit data.

[0066] Figure 2 This paper demonstrates a near-memory computing scheme using a multi-port parallel read / write high real-time near-memory processor. It utilizes custom memory units. The first to fourth near-memory computing cores are represented as near-memory cores 1 to 4 in the diagram. The unit read / write connections are connected to near-memory cores 1 to 4, allowing each core to perform independent read / write operations. This effectively handles computational scenarios involving dynamically updated weights and multiplexed inputs. The weight buffer in the diagram is... Figure 1 The data buffer in the chip. The four cores can pipeline and simultaneously perform one layer of neural network computation, thereby quadrupling the overall throughput.

[0067] Figure 3 This refers to the self-attention calculation process in the transformer algorithm. K W Q W V The static weights are multiplied by the input matrix X to form matrices K, Q, and V. Matrix K is transposed and multiplied by Q to obtain matrix A. A is then subjected to a softmax operation to obtain A'. Finally, A' is multiplied by V to obtain the final output result O.

[0068] Figure 4 Taking the self-attention calculation process in the Transformer algorithm as an example, this paper demonstrates the specific computational process of nonlinear operations in a multi-port parallel read / write high real-time near-memory processor. The workflow is as follows: The static weights (W...) are... K W Q W V After input X is written to the common storage area, the weights are then read into the computing cores Core1-3. The computing cores access the common storage area to obtain X, and simultaneously calculate K, Q, and V, as follows: Figure 4 As shown in a, 4b, and 4c; since K needs to be calculated... T For *Q and A'*V, the results of Q and V are directly stored in Core 2 and 3 as weights, while the result of K needs to be written back to the common storage area and then read out using the row and column transpose method into the core storing Q as input for calculation. Figure 4 As shown in d. K is obtained. T After *Q, a non-linear operation is performed within this core to obtain A', which is then shared with Core3 via the common memory area for V*A' calculation, finally yielding the output result O, such as... Figure 4 As shown in e.

[0069] The chip is manufactured using a 28nm CMOS process and can operate at 0.75-0.95V and 0-500MHz. Peak energy efficiency is achieved at 0.75V, with INT16 reaching 6.3 TOPS / W, INT8 reaching 25.4 TOPS / W, and INT4 reaching 101.4 TOPS / W.

[0070] In summary, this design presents a multi-port parallel read / write high real-time near-memory processor. It employs a multi-core shared memory near-memory design, a custom atomic-level instruction architecture, and a custom real-time variable-precision multiply-accumulate unit. It can perform vector inner products of various precisions, such as 4-bit, 8-bit, and 16-bit, without configuration. Thus, it can still complete multiply-accumulate operations in a single clock cycle even at high clock frequencies, thereby providing extremely high real-time performance.

[0071] The specific descriptions or embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of the invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time multi-port parallel read / write near-memory processor, comprising a data port and a control port, wherein the data port and the control port are connected to a near-memory processor module, characterized in that, It also includes a near-memory processor module and shared memory; The near memory processor module includes multiple near memory computing cores, each connected to the shared memory, and can refresh the data in the data cache and read and write data in the shared memory in real time. Each near-memory computing core includes a computing module, which in turn includes a multiply-accumulator. The multiply-accumulator includes a real-time variable-precision multiply-accumulator with an iterative structure. The multiply-accumulator combines the partial product results of different bit widths through an adder, which can adapt to the calculation of the inner product of vectors with different bit widths. The control port inputs atomic-level instructions and controls the computing modules to form a variety of rich operators to support neural network operations other than multiplication and addition; the shared memory includes a multi-port unit, a memory unit, and unit logic control; the near-memory computing core also includes a data cache, an instruction storage space, and a memory array read / write controller. The data buffer caches data for the computing module and the memory array read / write controller to read; the data buffer is connected to the data port; The data port is the location for data to enter and exit, and each near-memory computing core inputs or outputs several bits of data each time. The instruction storage space includes an instruction buffer, an instruction fetching module, a value retrieval logic module, and an instruction decoder; The instruction decoder controls the data buffer, computing module, and memory array read / write controller via a connection; the control port is connected to the instruction storage space of the near-memory computing core. The memory array read / write controller controls the flow of data, controls the position of the write unit in the shared memory, and controls the direction of data flow when it is read out. The shared memory includes several memory units, and each memory unit has multiple multi-port units; The function of the shared memory is to store data for the near-memory processor to read and write; The multi-port unit includes a unit logic control module and several unit modules.

2. The real-time multi-port parallel read / write near-memory processor according to claim 1, characterized in that, The connection is a one-way connection initiated by the instruction decoder in the instruction storage space; The computing module also includes a divider, a comparator, an adder, and a bit manipulator; the computing module exchanges data with the data buffer and the memory array read / write controller; the data buffer and the memory array read / write controller exchange data through a bidirectional connection; The memory array read / write controller is connected to the shared memory and exchanges data with it. The computing module performs various operations on the input data and outputs the results to the data buffer and the memory array read / write controller; the input data refers to the data flowing into the instruction decoder, data buffer, and memory array read / write controller connected to the computing module.

3. A real-time multi-port parallel read / write near-memory processor according to claim 1, characterized in that, The near-memory processor includes read mode and write mode; The read mode involves reading data stored in the shared memory and processing it in the near-memory processor module. The input signal enters the instruction storage space through the control port, and then controls the memory array read / write controller. Based on the location information of the multi-port unit in the memory cell to be read contained in the input signal, the data is transmitted to the shared memory. After receiving the location information to be read, the shared memory controls the corresponding memory cell to respond, and the cell logic controls the output of data at that location. The output data is stored in the data buffer or directly output from the data port. When the control port loads the instruction set compiled by the assembler into the instruction storage space of the arithmetic core, the operation of the arithmetic core is controlled through instruction decoding. When the calculation module is called to perform the calculation, the output information enters the calculation module to perform the corresponding instruction calculation. After the calculation is completed, the data may enter the data buffer or be directly output from the port. In the write mode, the input signal enters the data buffer from the data port or the data itself is stored in the data buffer; then the data in the data buffer enters the memory array read / write controller and is written to the corresponding shared memory unit; or, the input signal loads the instruction set compiled by the assembler into the instruction storage space of the arithmetic core, controls the operation of the arithmetic core through instruction decoding, calls the computing module to perform the operation, and then passes it to the data buffer to control the read / write array to write the corresponding data to the shared memory.

4. A real-time multi-port parallel read / write near-memory processor according to claim 3, characterized in that, The location information of the multi-port unit in the memory cell to be read is L-bit location information.

5. The multi-port parallel read / write near-memory processor according to claim 1 or 4, characterized in that, The plurality of near-memory computing cores are N, denoted as the first near-memory computing core to the Nth near-memory computing core; and N is greater than or equal to 4 and less than or equal to 16; each near-memory computing core inputs or outputs several bits of data, which are L bits, and the bit width of the near-memory computing core is L bits; the L bits are greater than or equal to 4 and less than or equal to 64.

6. The multi-port parallel read / write near-memory processor according to claim 5, characterized in that, The data port can input or output a maximum of L*N bits of data.

7. The multi-port parallel read / write near-memory processor according to claim 1, characterized in that, The number of memory units is Z; Z is greater than or equal to 4 and less than or equal to 8; the number of the multiple multi-port units is M; the value of M is greater than 1k and less than 4k.

8. The multi-port parallel read / write near-memory processor according to claim 1 or 4, characterized in that, The number of several unit modules is L, which is the same as the location information of the multi-port unit in the memory unit and the bit width of the near-memory computing core; denoted as CELL0 to CELL1. L-1 Each multi-port unit controls one L-bit of data; each near-memory computing core is connected to all memory units for reading and writing L-bit data.

Images