Transformer accelerator architecture based on monolithic three-dimensional integration

Abstract

The present disclosure relates to the technical field of artificial intelligence algorithm hardware acceleration, and in particular to a Transformer accelerator architecture based on monolithic three-dimensional integration, comprising a silicon-based logic circuit, used for controlling a current flowing direction of Transformer for logic operation; an RRAM-CIM array, used for executing a linear transformation operation; and a CFET 2T0C-CIM array, used for executing a matrix multiplication operation.

Description

Transformer accelerator architecture based on monolithic three-dimensional integration

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to Chinese Patent Application No. 202410595596.8, filed on May 14, 2024, entitled “Transformer accelerator architecture based on monolithic three-dimensional integration” and assigned to Tsinghua University. TECHNICAL FIELD

[0003] The present disclosure relates to the field of artificial intelligence algorithm hardware acceleration, and particularly relates to a Transformer accelerator architecture based on monolithic three-dimensional integration. BACKGROUND

[0004] Traditional silicon-based chips can only manufacture one layer of transistors due to the integration process. With the miniaturization of transistors, traditional chips face two major problems: the Von Neumann bottleneck and the slowdown of Moore's Law.

[0005] The Von Neumann bottleneck, also known as the Memory Wall, refers to the situation in computer programs where the speed of memory access is much slower than the speed of processor execution, causing the processor to wait for data loading or storage operations to complete, thereby affecting the efficiency of program execution. In simple terms, the speed difference between the processor and the memory causes a bottleneck in computing efficiency. This situation usually occurs when there are a large number of memory access operations in the program, and the execution speed of the processor is much faster than the memory access speed.

[0006] The slowdown of Moore's Law describes the weakening of the core driving force of the semiconductor and even the electronics industry. Moore's Law is an empirical rule that predicts that the performance of semiconductors will double every 18 to 24 months. However, in the past few years, the doubling speed of the number of transistors has changed from every 18-24 months to every 2-3 years, which means that the prediction of Moore's Law may fail in the next few years. The reasons for the slowdown and failure of Moore's Law are mainly two: one is the slowdown of technological progress, as the growth rate of the number of transistors approaches the physical limit, it may require higher costs to manufacture smaller and more powerful chips, which will lead to an increase in the price of computer hardware; two is the stability and reliability of the circuit, placing more transistors on integrated circuits requires smaller size and higher energy density, which may cause instability and reliability problems of the circuit.

[0007] In the field of Transformer deep learning, since Vaswani et al. first proposed in the paper "Attention Is All You Need" in 2017, it has been widely used in machine translation, text generation, language understanding and other natural language processing (NLP) tasks, and has also promoted a series of innovative model architectures such as BERT, GPT series, T5, etc.

[0008] As shown in FIG. 1, the Transformer is composed of two main parts, the encoder and the decoder, and its core is to use the multi-head self-attention mechanism to weight different parts of the input sequence, so as to process data at all sequence positions simultaneously.

[0009] However, the multi-head self-attention mechanism focuses on the linear transformation calculation of the fully weight stationary (FWS) module and the matrix multiplication calculation of the partially weight stationary (PWS) module. The fully weight stationary module can store weights for a long time, and can efficiently complete in-memory calculation using Kirchhoff's current law and Ohm's law, but it is difficult to rewrite the part of the weights (or parameters) that have been fixed after training and no longer updated with the changes of input data. The partially weight stationary module can store weights for a certain period of time, and some weights need to be updated or adjusted at runtime, so the weight rewriting is very simple, but the demand for dynamic update may require higher working frequency and computing resources, thus reducing the energy efficiency and speed. As can be seen, although the Transformer uses both the fully weight stationary module and the partially weight stationary module, it can greatly reduce the data transfer volume and overcome certain von Neumann bottlenecks, but the data transfer between them will still limit the system performance. SUMMARY

[0010] The present disclosure provides a monolithic three-dimensional integrated Transformer accelerator architecture to solve the problem that the data transfer between the fully weight stationary module and the partially weight stationary module in the existing Transformer accelerator structure will limit the system performance and make it difficult to completely overcome the two major problems of von Neumann bottleneck and Moore's law slowdown.

[0011] The first aspect embodiment of the present disclosure provides a single-chip three-dimensional integrated Transformer accelerator architecture, comprising: a silicon-based logic circuit, configured to control the current flow direction of a target Transformer to perform logical operation according to a target natural language processing task; an RRAM-CIM array stacked above the silicon-based logic circuit and interconnected with the silicon-based logic circuit through at least one interlayer dielectric via for communication, configured as a full static weight module of the target Transformer to perform linear transformation operation according to the target natural language processing task; and a CFET 2T0C-CIM array stacked above the RRAM-CIM array and interconnected with the RRAM-CIM array through at least one interlayer dielectric via for communication, configured as a semi-static weight module of the target Transformer to perform matrix multiplication operation according to the target natural language processing task.

[0012] In some embodiments, the RRAM-CIM array comprises a plurality of arrays of variable resistance nonvolatile memories and a first in-memory computing unit.

[0013] In some embodiments, the first in-memory computing unit comprises:

[0014] A first mapping subunit is configured to map data to be stored in the target natural language processing task as the resistance of each nonvolatile memory.

[0015] A second mapping subunit is configured to map a preset input vector in the target natural language processing task as a read voltage of each row of the RRAM-CIM array.

[0016] A computing subunit is configured to calculate an output current of each column of the RRAM-CIM array according to the resistance and the read voltage to complete the linear transformation operation.

[0017] In some embodiments, the CFET 2T0C-CIM array comprises a CFET 2T0C structure and a second in-memory computing unit, wherein,

[0018] The CFET 2T0C structure is configured to store target weights in the target natural language processing task to directly participate in the operation process of the second in-memory computing unit.

[0019] The second in-memory computing unit comprises a serial adder and a NOR gate, configured to multiply a preset input vector in the target natural language processing task with the target weights to obtain a plurality of partial products, and to accumulate the plurality of partial products through the serial adder and the NOR gate to output a multiplication result, so as to complete the matrix multiplication operation.

[0020] In some embodiments, the CFET 2T0C structure comprises a low leakage transistor as a write transistor and a vertical complementary field effect transistor as a read transistor.

[0021] In some embodiments, a drain of the low leakage transistor is connected to a common gate of the vertical complementary field effect transistor to write the target weight into a preset data source, a drain of a P-type transistor and a drain of an N-type transistor in the vertical complementary field effect transistor are connected to each other, a source of the P-type transistor is connected to a power supply terminal, and a source of the N-type transistor is connected to a ground to read the target weight from the preset data source.

[0022] The second aspect of the embodiments of the present disclosure provides a single-chip three-dimensional integrated Transformer accelerator architecture, comprising: a silicon-based logic circuit, configured to control a current flow direction of a target Transformer to perform logical operation according to a target natural language processing task; an RRAM-CIM array stacked above the silicon-based logic circuit and interconnected and communicated with the silicon-based logic circuit through at least one interlayer dielectric via, configured to serve as a full static weight module of the target Transformer to perform linear transformation operation according to the target natural language processing task; and an SRAM-CIM array stacked above the RRAM-CIM array and interconnected and communicated with the RRAM-CIM array through at least one interlayer dielectric via, configured to serve as a semi-static weight module of the target Transformer to perform matrix multiplication operation according to the target natural language processing task.

[0023] In some embodiments, the RRAM-CIM array comprises an array of a plurality of nonvolatile memories with variable resistance and a first in-memory computing unit.

[0024] In some embodiments, the first in-memory computing unit comprises:

[0025] a first mapping subunit, configured to map data to be stored in the target natural language processing task as a resistance value of each nonvolatile memory;

[0026] a second mapping subunit, configured to map a preset input vector in the target natural language processing task as a read voltage of each row of the RRAM-CIM array;

[0027] a computing unit, configured to calculate an output current of each column of the RRAM-CIM array according to the resistance value and the read voltage to complete the linear transformation operation.

[0028] In some embodiments, the SRAM-CIM array comprises an SRAM structure and a third in-memory computing unit, wherein,

[0029] the SRAM structure is configured to store target weights to directly participate in a calculation process of the third in-memory computing unit;

[0030] the third in-memory computing unit comprises a serial adder and a NOR gate, configured to multiply a preset input vector with the target weights to obtain a plurality of partial products, and to accumulate the plurality of partial products through the serial adder and the NOR gate to output a multiplication result, so as to complete the matrix multiplication operation.

[0031] The Transformer accelerator architecture based on monolithic three-dimensional integration provided by the embodiments of the present disclosure utilizes the high-density interlayer medium via of the monolithic three-dimensional integration technology to stack the silicon-based logic circuit, the RRAM-CIM array and the CFET 2T0C-CIM macro unit to realize high-density and low-latency inter-module communication, so that each module can avoid disadvantages and respectively play the advantages of different in-memory computing to realize different functions, greatly reduce the data carrying amount, overcome the Von Neumann bottleneck, continue Moore's Law, and have better Transformer acceleration effect compared with various Baseline implementation schemes.

[0032] Additional aspects and advantages of the present disclosure will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0033] The above and / or additional aspects and advantages of the present disclosure will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0034] FIG. 1 is a structural schematic diagram of a Transformer model, wherein (a) is a stack of an Encoder and a Decoder, (b) is an Encoder structure, (c) is a Decoder structure, and (d) is a calculation in a multi-head self-attention mechanism;

[0035] FIG. 2 is a structural schematic diagram of a Transformer accelerator architecture based on monolithic three-dimensional integration provided by an embodiment of the present disclosure;

[0036] FIG. 3 is a schematic diagram of the working principle of a RRAM-CIM array provided by an embodiment of the present disclosure;

[0037] FIG. 4 is a structural schematic diagram of a CFET 2T0C-CIM array provided by an embodiment of the present disclosure, wherein (a) is a CFET 2T0C structure, and (b) is a CFET 2T0C-CIM array;

[0038] FIG. 5 is a structural schematic diagram of another Transformer accelerator architecture based on monolithic three-dimensional integration provided by an embodiment of the present disclosure;

[0039] FIG. 6 is a specific schematic diagram of a Transformer accelerator architecture based on monolithic three-dimensional integration provided by an embodiment of the present disclosure;

[0040] FIG. 7 is a manufacturing process schematic diagram of a Transformer accelerator architecture based on monolithic three-dimensional integration provided by an embodiment of the present disclosure;

[0041] FIG. 8 is a cross-sectional transmission electron microscope photo of a Transformer accelerator architecture based on monolithic three-dimensional integration provided by an embodiment of the present disclosure;

[0042] FIG. 9 is a principle schematic diagram of implementing in-memory computing by a full static weight module provided by an embodiment of the present disclosure;

[0043] FIG. 10 is a principle schematic diagram of implementing in-memory computing by a semi-static weight module provided by an embodiment of the present disclosure;

[0044] FIG. 11 is a process flow chart of completing Transformer computation by a Transformer accelerator architecture based on monolithic three-dimensional integration provided by an embodiment of the present disclosure;

[0045] FIG. 12 is a performance evaluation result schematic diagram of a Transformer accelerator architecture based on monolithic three-dimensional integration provided by an embodiment of the present disclosure.

[0046] Legend of reference signs: 100-Transformer accelerator architecture based on monolithic three-dimensional integration, 101-silicon-based logic circuit, 102-RRAM-CIM array, 103-CFET 2T0C-CIM array, 104-SRAM-CIM array. DETAILED DESCRIPTION

[0047] The Transformer accelerator architecture based on monolithic three-dimensional integration of an embodiment of the present disclosure is described below with reference to the accompanying drawings.

[0048] FIG. 2 is a structural schematic diagram of a Transformer accelerator architecture based on monolithic three-dimensional integration of an embodiment of the present disclosure.

[0049] As shown in FIG. 2, the Transformer accelerator architecture 100 based on monolithic three-dimensional integration includes a silicon-based logic circuit 101, an RRAM-CIM array 102, and a CFET 2T0C-CIM array 103.

[0050] The silicon-based logic circuit 101 is configured to perform logical operation according to the current flow direction of the target Transformer. The RRAM-CIM array 102 is stacked above the silicon-based logic circuit 101 and is interconnected and in communication with the silicon-based logic circuit 101 through at least one interlayer dielectric via, and is configured to serve as a full-static weight module of the target Transformer to perform linear transformation operation according to the target natural language processing task. The CFET 2T0C-CIM array 103 is stacked above the RRAM-CIM array 102 and is interconnected and in communication with the RRAM-CIM array 102 through at least one interlayer dielectric via, and is configured to serve as a semi-static weight module of the target Transformer to perform matrix multiplication operation according to the target natural language processing task.

[0051] In some embodiments, the RRAM-CIM array 102 includes a plurality of resistance-variable nonvolatile memory arrays and a first in-memory computing unit.

[0052] In some embodiments, the first in-memory computing unit includes:

[0053] A first mapping subunit is configured to map the to-be-stored data in the target natural language processing task to the resistance of each nonvolatile memory.

[0054] A second mapping subunit is configured to map a preset input vector in the target natural language processing task to the read voltage of each row of the RRAM-CIM array.

[0055] A computing subunit is configured to calculate the output current of each column of the RRAM-CIM array according to the resistance and the read voltage, so as to complete the linear transformation operation.

[0056] In actual execution, as shown in FIG. 3, the RRAM-CIM array 102 is configured to serve as a full-static weight module of the target Transformer, and the core unit of the RRAM-CIM array 102 is a storage array composed of resistance-variable nonvolatile memories. According to Kirchhoff's law, the first in-memory computing unit can complete the matrix-vector multiplication operation process. Specifically, the first mapping subunit is configured to map the to-be-stored data in the target natural language processing task to the resistance of each memory cell and write the resistance into the RRAM-CIM array 102, the second mapping subunit is configured to map the input vector to the read voltage of each row of the array, and finally the first computing subunit is configured to calculate and output the output current of each column, which is the product of the input voltage and the resistance-variable memory conductance, ∑I = ∑V * G.

[0057] In addition, the in-memory computing array based on Nand Flash, phase change memory (PCM) and the like also has the corresponding characteristics of the RRAM-CIM array, and therefore, the in-memory computing array based on Nand Flash, phase change memory (PCM) and the like can also be used as the full static weight module of the target Transformer by the person skilled in the art, and linear transformation operation can be performed according to the target natural language processing task.

[0058] In some embodiments, the CFET 2T0C-CIM array 103 includes a CFET 2T0C structure and a second in-memory computing unit, wherein,

[0059] The CFET 2T0C structure is used to store a target weight in a target natural language processing task to directly participate in the operation process of the second in-memory computing unit;

[0060] The second in-memory computing unit includes a serial adder and a NOR gate, which are used to multiply a preset input vector in the target natural language processing task with the target weight to obtain a plurality of partial products, and add the plurality of partial products through the serial adder and the NOR gate to output a multiplication result, so as to complete the matrix multiplication operation.

[0061] As shown in FIG. 4, the CFET 2T0C structure includes a low-leakage transistor and a vertical complementary field effect transistor, wherein the low-leakage transistor is used as a write tube, and the low-leakage transistor can be IGZO-NFET and the like, the vertical complementary field effect transistor is used as a read tube, and the vertical complementary field effect transistor can be CFET composed of CNT transistor and IGZO transistor and the like, the drain of the low-leakage transistor is connected with the common gate of the vertical complementary field effect transistor, so as to write the target weight into the preset data source, the drain of the P-type transistor and the drain of the N-type transistor in the vertical complementary field effect transistor are connected with each other, the source of the P-type transistor is connected with a power supply end, and the source of the N-type transistor is connected with a ground wire, so as to read the target weight from the preset data source.

[0062] In actual execution, since the output of the traditional 2T0C structure is current, if it is used in digital in-memory computing, it still needs to pass through a current-voltage converter, which increases the circuit complexity and cost. Therefore, the CFET 2T0C structure is selected as the semi-static weight module of the target Transformer in the embodiments of the present disclosure. The CFET 2T0C structure uses CFET as a read tube, and the output is voltage, which can directly participate in digital in-memory computing, and can realize the operation of the partial product in multiplication. Specifically, the CFET 2T0C structure first stores the target weight in the target natural language processing task, and then inputs another multiplier by using the second in-memory computing unit to obtain the partial product, and finally outputs the partial product to the serial adder and the NOR gate to obtain the product of multiplication, and complete the matrix multiplication operation.

[0063] As shown in FIG. 5, the in-memory computing array based on SRAM, DRAM, etc. also has corresponding characteristics, therefore, another Transformer accelerator architecture based on monolithic three-dimensional integration is proposed in the embodiment of the present disclosure, which includes a silicon-based logic circuit 101, a RRAM-CIM array 102 and an SRAM-CIM array 104, that is, the CFET 2T0C-CIM array 103 is replaced by the SRAM-CIM array 104, the SRAM-CIM array 104 is stacked above the RRAM-CIM array 102 and is interconnected and communicated with the RRAM-CIM array through at least one interlayer dielectric via, used as a semi-static weight module of the target Transformer, to perform matrix multiplication operation according to the target natural language processing task.

[0064] In some embodiments, the SRAM-CIM array includes an SRAM structure and a fourth in-memory computing unit, wherein,

[0065] the SRAM structure is used to store target weights to directly participate in the calculation process of the third in-memory computing unit;

[0066] the fourth in-memory computing unit includes a serial adder and a NOR gate, used to multiply a preset input vector with the target weights to obtain a plurality of partial products, and to accumulate the plurality of partial products through the serial adder and the NOR gate to output a multiplication result, so as to complete the matrix multiplication operation.

[0067] In actual execution process, the SRAM structure first stores the target weights in the target natural language processing task, and then inputs another multiplier by using the second in-memory computing unit to obtain a partial product, and finally obtains the product of multiplication by outputting the partial product to the serial adder and the NOR gate, to complete the matrix multiplication operation.

[0068] It should be noted that the difference between the SRAM structure and the CFET 2T0C structure is that the SRAM structure is static storage, without refresh circuit, and the data can be kept all the time; the CFET 2T0C structure is dynamic storage, which needs a refresh circuit to keep the data, but in the PWS application facing the Transformer proposed in the embodiment of the present disclosure, since the operation of the partial product can be completed within the holding time of the CFET 2T0C, the refresh circuit is not needed.

[0069] In actual execution process, as shown in FIGS. 6 and 8, the Transformer accelerator architecture based on monolithic three-dimensional integration integrates the silicon-based logic circuit 101, the RRAM-CIM array 102 and the CFET 2T0C-CIM array 103 together to manufacture a prototype chip, the RRAM-CIM array 102 realizes linear transformation (X, W Q ,WK W V ->Q,K,V), after the training of the Transformer model, W Q , W K , W V In inference, it remains unchanged, and the calculation method is completely similar to the matrix-vector multiplication of the neural network, so that the CFET 2T0C-CIM array 103 can efficiently implement matrix multiplication operations (Q,K->QK,V->QKV). Because Q, K, and V change according to different inputs X every time, the in-memory calculation is performed using the semi-static weight module, thereby efficiently implementing matrix multiplication operations. The manufacturing sequence is as follows: the first layer manufactures the silicon-based part, such as the process flow of the black part in FIG. 7, which is implemented using a standard 130 nm Si process for the logic part, the second layer manufactures the RRAM-CIM array, such as the process flow of the dark gray part in FIG. 7, which needs to meet the thermal budget condition (<300°C) of the subsequent process, for the implementation of the full static weight module, and the third layer manufactures the CFET 2T0C-CIM macro unit, such as the process flow of the light gray part in FIG. 7, which needs to meet the thermal budget condition (<300°C) of the subsequent process, for the implementation of the semi-static weight module.

[0070] The functions of the full static weight module based on the RRAM-CIM array 102 to implement linear transformation and the functions of the semi-static weight module based on the CFET 2T0C-CIM array 103 to implement matrix multiplication are verified as follows.

[0071] As shown in FIG. 9, the full static weight module based on the RRAM-CIM array 102 is used to implement linear transformation, that is, in-memory calculation of matrix-vector multiplication. First, the weights are stored on the full static weight module (FWS) by programming, in which the element values in the weight matrix are mapped into the conductance values of the RRAM and stored in the RRAM array, which remain unchanged in subsequent multiple inference processes. Then, in each inference, the input vector (X) is mapped into the voltage input of the RRAM array, in which the input voltage of each bit line is proportional to the element value in the vector. Subsequently, according to Ohm's law and Kirchhoff's current law, the output current of the source line should be proportional to the product of the weight matrix and the input vector, that is, the result of matrix-vector multiplication (MVM result). FIG. 9 shows the demonstration results on the 128k-bit RRAM array of the prototype chip, which counts the expected output and actual output of each element in the output vector and shows the probability distribution thereof, proving that the RRAM-CIM array can complete in-memory calculation of matrix-vector multiplication and avoid the transfer of the two-dimensional weight matrix, overcoming the von Neumann bottleneck, and using Kirchhoff's current law and Ohm's law to complete analog calculation, thereby having high calculation efficiency.

[0072] As shown in FIG. 10, the in-memory computing of matrix multiplication is implemented by using the semi-static weight module based on the CFET 2T0C-CIM array 103, and the essence of matrix multiplication is the multiplication of two numbers A and B, and the implementation process is as follows: first, the multiplier A is stored in the CFET 2T0C-CIM array, then the multiplier B is input serially and broadcast to different bits to obtain the partial product of multiplication of any two bits, and then the partial products are input into the serial adder to obtain the final product of A and B.

[0073] Therefore, as shown in FIG. 11, the single three-dimensional integrated mode is used to stack the silicon-based logic circuit, the RRAM-CIM array and the CFET 2T0C-CIM macro unit in the embodiment of the disclosure, the RRAM-CIM is used to implement the full-static weight module (linear transformation operation), the CFET 2T0C-CIM is used to implement the semi-static weight module (matrix-matrix multiplication), the silicon-based part is used to implement the logic control, the advantages of different in-memory computing are used to implement different functions, the single three-dimensional integrated architecture is used to increase the bandwidth, the data transfer amount is greatly reduced, the von Neumann bottleneck is overcome, and the Transformer acceleration is better implemented.

[0074] As shown in FIG. 12, since the matrix multiplication (MatMul) is changed at each inference, the two inputs are changed, and therefore the full-static weight module (FWS) is used for calculation, which can cause a large delay due to the low efficiency of programming / data storage, and therefore the static weight unit (FWS only) is not completely implemented in the embodiment of the disclosure, and the RRAM-CIM array is used to reduce the time consumption of matrix multiplication, and the disclosure (M3D-SFP) is 1.9 times faster than the existing FWS only; since the delay of the semi-static weight module (PWS) in completing the linear transformation (Linear) is much higher than that of the full-static weight module (FWS), and therefore the semi-static weight module (PWS only) is not completely implemented, and the CFET 2T0C-CIM array is used to reduce the time consumption of linear transformation, and the disclosure (M3D-SFP) is 2.6 times faster than the existing PWS only; the single three-dimensional integration is used to accelerate the data transfer between the system function modules such as the semi-static weight module and the full-static weight module, the high-density interlayer medium via hole of the single three-dimensional integration is used for high-bandwidth communication, the von Neumann bottleneck is overcome, and the disclosure (M3D-SFP) is 12.6 times faster than the two-dimensional mode.

[0075] In summary, according to the single-chip three-dimensional integrated Transformer accelerator architecture proposed in the embodiments of the present disclosure, the single-chip three-dimensional integrated technology is used to stack the silicon-based logic circuit, the RRAM-CIM array and the CFET 2T0C-CIM macro unit by using the high-density interlayer medium via hole to realize high-density and low-delay inter-module communication, so that each module can avoid the disadvantages and play the advantages of different in-memory computing to realize different functions, greatly reduce the data carrying amount, overcome the von Neumann bottleneck, continue Moore's law, and have better Transformer acceleration effect compared with various Baseline implementation schemes.

[0076] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0077] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

Claims

1. A monolithic three-dimensional integrated based Transformer accelerator architecture, wherein, Comprising: a silicon-based logic circuit for performing logical operations according to a target natural language processing task to control the current flow direction of a target Transformer; an RRAM-CIM array stacked above the silicon-based logic circuit and interconnected in communication with the silicon-based logic circuit through at least one interlayer dielectric via, for serving as a full-static weight module of the target Transformer to perform a linear transformation operation according to the target natural language processing task; a CFET 2T0C-CIM array stacked above the RRAM-CIM array and interconnected in communication with the RRAM-CIM array through at least one interlayer dielectric via, for serving as a semi-static weight module of the target Transformer to perform a matrix multiplication operation according to the target natural language processing task.

2. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 1, wherein, The RRAM-CIM array comprises a plurality of arrays of non-volatile memories with variable resistance and a first in-memory computing unit.

3. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 2, wherein, The first in-memory computing unit comprises: a first mapping subunit for mapping data to be stored in the target natural language processing task as the resistance of each non-volatile memory; a second mapping subunit for mapping a preset input vector in the target natural language processing task as the read voltage of each row of the RRAM-CIM array; a computing subunit for calculating the output current of each column of the RRAM-CIM array according to the resistance and the read voltage to complete the linear transformation operation.

4. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 1, wherein, The CFET 2T0C-CIM array comprises a CFET 2T0C structure and a second in-memory computing unit, wherein, the CFET 2T0C structure is used to store target weights in the target natural language processing task to directly participate in the operation process of the second in-memory computing unit; the second in-memory computing unit comprises a serial adder and a NOR gate, for multiplying a preset input vector in the target natural language processing task with the target weights to obtain a plurality of partial products, and adding the plurality of partial products through the serial adder and the NOR gate to output a multiplication result to complete the matrix multiplication operation.

5. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 4, wherein, The CFET 2T0C structure comprises a low-leakage transistor and a vertical complementary field effect transistor, wherein the low-leakage transistor serves as a write tube, and the vertical complementary field effect transistor serves as a read tube.

6. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 5, wherein, The drain of the low-leakage transistor is connected with the common gate of the vertical complementary field effect transistor to write the target weights into a preset data source, the drain of the P-type transistor and the drain of the N-type transistor in the vertical complementary field effect transistor are connected with each other, the source of the P-type transistor is connected with a power supply end, and the source of the N-type transistor is connected with a ground wire to read the target weights from the preset data source.

7. A monolithic three-dimensional integrated based Transformer accelerator architecture, wherein, Comprising: a silicon-based logic circuit for performing logical operations according to a target natural language processing task to control the current flow direction of a target Transformer; An RRAM-CIM array is stacked above the silicon-based logic circuit and is interconnected in communication with the silicon-based logic circuit through at least one interlayer dielectric via, for serving as a full-static weight module of the target Transformer, to perform a linear transformation operation according to the target natural language processing task. An SRAM-CIM array is stacked above the RRAM-CIM array and is interconnected in communication with the RRAM-CIM array through at least one interlayer dielectric via, for serving as a semi-static weight module of the target Transformer, to perform a matrix multiplication operation according to the target natural language processing task.

8. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 7, wherein, The RRAM-CIM array includes a plurality of arrays of variable resistance nonvolatile memories and a first in-memory computing unit.

9. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 8, wherein, The first in-memory computing unit includes: A first mapping subunit is configured to map data to be stored in the target natural language processing task to a resistance value of each nonvolatile memory. A second mapping subunit is configured to map a preset input vector in the target natural language processing task to a read voltage of each row of the RRAM-CIM array. A computing unit is configured to calculate an output current of each column of the RRAM-CIM array according to the resistance value and the read voltage, to complete the linear transformation operation.

10. The monolithic three-dimensional integrated based Transformer accelerator architecture of claim 7, wherein, The SRAM-CIM array includes an SRAM structure and a third in-memory computing unit, wherein: The SRAM structure is configured to store target weights to directly participate in a calculation process of the third in-memory computing unit. The third in-memory computing unit includes a serial adder and a NOR gate, configured to multiply a preset input vector and the target weights to obtain a plurality of partial products, and to accumulate the plurality of partial products through the serial adder and the NOR gate, to output a multiplication result, to complete the matrix multiplication operation.

Images