Signed multiplication circuit, column-level MAC circuit, maximum value search circuit and chip

Abstract

The present invention belongs to the field of integrated circuits, and specifically relates to a signed multiplication circuit, a column-level MAC circuit, a maximum value search circuit and a chip. The signed multiplication circuit includes a numerical operation unit and a symbol operation unit; the numerical operation unit is composed of at least one SRAM unit with read-write separation and a dual-port read port. The symbol operation unit is composed of three AND gates and a D flip-flop. The sign bit operation unit is used to transfer the numerical bit of the operand to the numerical operation unit according to the product of the sign bit, and complete the multiplication operation between the numerical bits. The product result is finally reflected in the discharge state of the bit line. A column-level MAC circuit can be formed by using multiple signed multiplication circuits, and a maximum value search circuit of the MAC result can be obtained by arraying the signed multiplication circuit. The maximum value search circuit of the present invention can solve the problem that the existing in-memory computing architecture is difficult to accelerate the neural network computing task with a self-attention mechanism.

Description

Technical Field

[0001] The present invention belongs to the field of integrated circuits, and in particular relates to a signed multiplication circuit, a column-level MAC circuit, a MAC operation maximum value finding circuit, and a neural network accelerator chip. Background Art

[0002] In today's era, artificial intelligence technology is developing at an astonishing speed. These edge devices are facing an increasingly massive amount of data processing tasks, and traditional computing architectures are gradually exposing many serious limitations when dealing with such huge data processing pressure. As a new storage technology, in-memory computing has effectively broken through the "storage wall" dilemma faced by the traditional von Neumann computing architecture. Its core essence is that computing operations can be performed directly inside the memory. In the traditional computing architecture, the memory and computing units are separated from each other. In-memory computing technology boldly innovates this traditional model and cleverly integrates computing operations into the memory. In this way, not only the data processing method is changed, but also the computing efficiency is significantly improved, bringing new development opportunities to the computing field. In recent years, with the rapid development of science and technology, in-memory computing technology has also developed rapidly in fields such as artificial intelligence and visual recognition, which also symbolizes that the performance requirements for in-memory computing architecture in scientific research are becoming more and more stringent.

[0003] In-memory computing is mostly used in neural network accelerators, and the main operation of neural network accelerators is matrix operations. Recently, in-memory computing chips based on various algorithms, such as convolutional neural networks (CNNs) and Transformer, have developed on a large scale. Since the in-memory computing circuits strictly correspond to the mapping of the algorithms, the structure and macro-architecture of the memory chip have very high requirements for the chip's functions. As the algorithms continue to change and the functionality continues to improve, the performance and functional requirements for in-memory computing to implement matrix operations are also increasing.

[0004] Transformer has a unique self-attention mechanism and extremely high computing accuracy. It has broken the monopoly of traditional convolutional neural networks (CNNs) in the field of artificial intelligence (AI) and has become a research focus in the field of in-memory computing. In the computing mechanism of neural network accelerators, when the self-attention mechanism is implemented, it is necessary to determine the position of the column where the maximum value is located after the input vector is multiplied by the hash matrix in the hash bucketing stage, so as to know the bucket to which the input vector is assigned. Since the hash matrix is ​​composed of the same positive and negative matrices, it is only necessary to calculate the maximum absolute value of the multiplication result of the input vector and half of the hash matrix to determine the position of the bucket to which the input vector is assigned while saving a lot of area. However, the existing technology lacks a system architecture for in-memory computing circuits that can perform the task of finding the column where the absolute value maximum is located, which results in the existing various types of neural network accelerators needing to spend a lot of memory array area to implement similar operations; and the efficiency and power consumption of the operation are also difficult to meet expectations. Summary of the invention

[0005] In order to solve the problem that the existing in-memory computing architecture is difficult to accelerate the neural network operation tasks with a self-attention mechanism, the present invention provides a signed multiplication circuit, a column-level MAC circuit, a MAC operation maximum value search circuit and a neural network accelerator chip.

[0006] The technical solution provided by the present invention is:

[0007] A signed multiplication circuit is used to perform a multiplication operation between a positive operand and a signed weight. The signed multiplication circuit includes a numerical operation unit and a sign operation unit. The numerical operation unit includes at least one XT unit; the XT unit adopts an SRAM unit with read-write separation and a read dual port. The read port of the XT unit is composed of two NMOS tubes N3 and N4; N3 is used as a transmission tube between the storage node Q and the read bit line BL; N4 is used as a transmission tube between the storage node QB and the read bit line BLB.

[0008] The symbol operation unit is composed of three AND gates AND1, AND2, and AND3 and a D flip-flop D1. The circuit connection relationship is: the output end of AND1 is connected to the D end of D1; the positive phase output end Q1 of D1 is connected to one of the input ends of AND2; the negative phase output end Q2 is connected to one of the input ends of AND3. The other output ends of AND2 and AND3 are both connected to the operation enable signal EN. The output end of AND2 is connected to the gate of N4 in each XT unit; the output end of AND3 is connected to the gate of N3 in each XT unit.

[0009] Among them, the two input ends of AND1 are used to input the sign bits S1 and S2 of the operand and the weight respectively; the enable signal EN is used to input the numerical bit N1 of the operand using pulse width encoding; the storage node in each XT unit is used to pre-store one of the numerical bits of the weight; the product result is reflected in the level change of the read bit lines BL and BLB.

[0010] In the signed multiplication circuit provided by the present invention, the encoding rule of the sign bit of the operand and the weight is:

[0011] When S1=1, it means the sign bit of the operand is “+”; when S1=0, it means the sign bit of the operand is “-”.

[0012] When S2=1, it means the sign bit of the weight is “+”; when S2=0, it means the sign bit of the operand is “-”.

[0013] Furthermore, the encoding rules for the numerical bits of operands and weights are:

[0014] The unit pulse width △t is defined to represent the value of the operand as 1, and the multiple of the high-level pulse width of S1 input relative to the unit pulse width △t is the decimal value of the operand.

[0015] When Q=1, QB=0, it means that the numerical value of the weight is "1"; when Q=0, QB=1, it means that the numerical value of the weight is "0".

[0016] In the signed multiplication circuit provided by the present invention, the read bit lines BL and BLB are both precharged to a high level before the operation. After the operation is completed, if the level of BL decreases, it means that the product result is a negative number, and the ratio of the level decrease of BL relative to the unit decrease △V is the value of the product result; if the level of BLB decreases, it means that the product result is a positive number, and the ratio of the level decrease of BLB relative to the unit decrease △V is the value of the product result.

[0017] As a further improvement of the present invention, in the numerical operation unit, when a plurality of XT units are included, each XT unit shares the same set of read bit lines BL and BLB.

[0018] When it is necessary to complete the multiplication operation of a (N+1)-bit signed weight and a signed operand, N XT units need to be enabled; each XT unit is used to pre-store the values ​​of different weight bits on the numerical bits of the multi-bit weight; the MOS tubes used by each XT unit with different bit weights have different width-to-length ratios.

[0019] As a further improvement of the present invention, the XT unit in the numerical operation unit adopts a read-write separated 8T-SRAM unit; the 8T-SRAM unit is composed of two PMOS tubes P1 and P2 and six NMOS tubes N1 to N6; the circuit connection relationship is:

[0020] P1, P2, N1, and N2 are cross-coupled in reverse phase to form a latch structure including storage nodes Q and QB. N3 and N4 form a read channel; N3 is used as a transmission tube between storage node Q and read bit line BL; N4 is used as a transmission tube between storage node QB and read bit line BLB. N5 and N6 form a write channel; N5 is used as a transmission tube between storage node Q and write bit line TBL; N6 is used as a transmission tube between storage node QB and write bit line TBLB; the gates of N5 and N6 are connected to write word line TWL.

[0021] The present invention also includes a column-level MAC circuit, which is composed of a plurality of signed multiplication circuits as described above arranged in columns; each signed multiplication circuit shares the same set of read bit lines BL and BLB.

[0022] In the column-level MAC circuit, each signed multiplication circuit is used to synchronously execute the tasks of each multiplication operation in the MAC operation; then the sum of all positive terms included in the MAC operation is reflected on the read bit line BLB, and the sum of all negative terms included in the MAC operation is reflected on the read bit line BL.

[0023] As a further improvement of the present invention, the column-level MAC circuit also includes or is connected to a quantization circuit. The quantization circuit is mainly composed of two analog-to-digital converters ADC1 and ADC2, and a signed adder ADD. ADC1 and ADC2 are used to quantize the operation results on BL and BLB respectively. The outputs of ADC1 and ADC2 serve as two inputs of ADD; the output of ADD is the final result of the MAC operation.

[0024] The present invention also includes a MAC operation maximum value finding circuit, which is used to find the operation task with the largest absolute value of the operation result among multiple signed MAC operation tasks. The MAC operation maximum value finding circuit includes: a storage array, a maximum value finding module and a peripheral circuit.

[0025] The storage and calculation array is composed of a plurality of column-level MAC circuits as described above arranged in rows; each column in the storage and calculation array is used to perform an independent MAC calculation task.

[0026] The maximum value search module includes a sense amplifier array and a decoder. The sense amplifier array is composed of a plurality of voltage latch sense amplifiers and a plurality of XOR gates corresponding to the number of columns of the storage and calculation array; the two input ports of each sense amplifier are respectively connected to the bit lines BL and BLB of one of the column-level MAC circuits in the storage and calculation array; the output terminals OUT and OUTB of each sense amplifier are connected to the two input terminals of XOR, and the output terminal of XOR is connected to the input terminal of the decoder. The decoder is used to latch the corresponding column that first generates the output signal in each XOR gate, and then generate a corresponding column code and output it.

[0027] The MAC operation maximum value finding circuit provided by the present invention is a CIM circuit designed based on an SRAM circuit, wherein the peripheral circuit includes other necessary circuits related to executing data storage and logic operations.

[0028] As a further improvement of the present invention, the sensitive amplifier in the MAC operation maximum value finding circuit is composed of 6 PMOS tubes PM1 to PM6 and 2 NMOS tubes NM1 and NM2. The circuit connection relationship is:

[0029] The sources of PM1~PM4 are connected to the power supply; the drains of PM1, PM2, NM1, and PM5 are connected to the gates of PM3 and NM2 and serve as the positive output node OUT. The drains of PM3, PM4, PM6, and NM2 are connected to the gates of PM2 and NM1 and serve as the negative output node OUTB. The sources of NM1 and NM2 are connected to the drain of NM3; the gates of PM5, PM6, and NM3 are connected to the enable signal SAE; the gates of PM1 and PM4 are connected to the pre-charge enable signal Pre; the source of PM5 is connected to BL; the source of PM6 is connected to BLB; the source of NM3 is grounded.

[0030] The present invention also includes a neural network accelerator chip, which is used to perform neural network computing tasks with a self-attention mechanism, and the chip adopts the MAC computing maximum value finding circuit as mentioned above.

[0031] The technical solution provided by the present invention has the following beneficial effects:

[0032] The present invention adds a sign operation unit composed of a D flip-flop and an AND gate on the basis of a read-write separation SRAM unit with a dual read port, and then designs a new circuit that can realize multiplication operation between signed input and signed weight. Based on the new signed multiplication circuit, the present invention further designs a new column-level MAC operation circuit that can output the sum of positive terms and the sum of negative terms on the output bit lines on both sides.

[0033] Combined with the newly designed column-level MAC operation circuit, the present invention further designs a circuit that can quickly identify the operation task with the maximum absolute value of the MAC result without quantizing the MAC result. By using this circuit designed based on the CIM architecture to find the maximum absolute value of the MAC result, the performance of the neural network accelerator is improved, so that the neural network accelerator can quickly process the data processing tasks of the neural network model with the self-attention mechanism.

[0034] Compared with the existing method of calculating in memory to locate the column with the maximum absolute value, the method of finding the column with the maximum absolute value by in-memory calculation proposed by the present invention can reduce a large amount of array area. It is of great significance for certain applications with high requirements for identifying the column with the maximum absolute value, such as signal recognition processing, intelligent visual detection, etc., and it also helps to expand the functions of the in-memory computing chip and improve the computing efficiency of the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a circuit diagram of a circuit for performing 2-bit weighted signed multiplication provided in Embodiment 1 of the present invention.

[0036] Figure 2 This is a circuit diagram of a circuit for performing signed multiplication of multi-bit weights provided in Embodiment 1 of the present invention.

[0037] Figure 3 This is a circuit diagram of a column-level MAC circuit provided in Embodiment 2 of the present invention.

[0038] Figure 4 It is a circuit diagram of a column-level MAC circuit including a quantization circuit provided in Embodiment 2 of the present invention.

[0039] Figure 5 This is a circuit diagram of a MAC operation maximum value finding circuit provided in Embodiment 3 of the present invention.

[0040] Figure 6 4 is a circuit diagram of a voltage latch type sense amplifier used in Example 3 of the present invention.

[0041] Figure 7 To verify the test results of the circuit in the experiment performing the signed multiplication task.

[0042] Figure 8 To verify the test results of the maximum value search task among the multi-MAC operation results of the circuit in the experiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] Example 1

[0045] This embodiment provides a signed multiplication circuit, which is used to perform a multiplication operation between a positive operand and a signed weight. The most obvious difference between the signed multiplication circuit provided by this embodiment and other circuits with the same function is that the circuit of this embodiment can output the operation result through different paths when the signs of the operation result are different. In the subsequent circuit design, the technicians also use this characteristic of the signed multiplication circuit to realize the rapid search for the operation task with the largest absolute value of the operation result in multiple MAC operations.

[0046] Specifically, Figure 1 As shown, the signed multiplication circuit provided in this embodiment includes a numerical operation unit and a sign operation unit. The numerical operation unit includes at least one SRAM unit with read-write separation and dual read ports. For the convenience of description, this embodiment will collectively refer to various SRAM units with read-write separation and dual read ports using different devices or circuit designs as XT units. In various XT units, the two NMOS tubes constituting the read port are defined as N3 and N4 respectively; wherein N3 is used as a transmission tube between the storage node Q and the read bit line BL; and N4 is used as a transmission tube between the storage node QB and the read bit line BLB.

[0047] In the typical solution provided in this embodiment, the XT unit in the numerical operation unit adopts a read-write separated 8T-SRAM unit. The 8T-SRAM unit is composed of two PMOS tubes P1 and P2 and six NMOS tubes N1 to N6; the circuit connection relationship is:

[0048] P1, P2, N1, and N2 are cross-coupled in reverse phase to form a latch structure including storage nodes Q and QB. N3 and N4 form a read channel; N3 is used as a transmission tube between storage node Q and read bit line BL; N4 is used as a transmission tube between storage node QB and read bit line BLB. N5 and N6 form a write channel; N5 is used as a transmission tube between storage node Q and write bit line TBL; N6 is used as a transmission tube between storage node QB and write bit line TBLB; the gates of N5 and N6 are connected to write word line TWL.

[0049] The operation logic of the 8T-SRAM unit to realize the storage function is as follows:

[0050] 1. Data writing

[0051] In the data write mode, the two NMOS transistors N3 and N4 of the read channel are in the off state, and the read bit lines BL and BLB are separated from the intermediate latch structure. At this time, using the two NMOS transistors N5 and N6 of the write channel and the two write bit lines TBL and TBLB, different data can be written in the latch structure (composed of P1, P2, N1, and N2) containing the storage nodes Q and QB.

[0052] For example, if "1" needs to be written in the storage node, the write bit line TBL is first set to a high level and TBLB is set to a low level; then the write word line TWL is set to a high level, at which time N5 and N6 are turned on; a charging path is formed between TBL and the storage node Q, and Q is pulled high; a discharge path is formed between TBLB and the storage node QB, and QB is pulled low. Finally, Q=1, QB=0, that is, the writing of data "1" is completed.

[0053] On the contrary, if "0" needs to be written in the storage node, the write bit line TBL needs to be set to a low level and TBLB needs to be set to a high level first; then the write word line TWL needs to be set to a high level, at which time N5 and N6 are turned on; a discharge path is formed between TBL and the storage node Q, and Q is pulled down to a low level; a charging path is formed between TBLB and the storage node QB, and QB is pulled up to a high level. Finally, Q=0, QB=1, and the writing of data "0" is completed.

[0054] (II) Data reading

[0055] In the data read mode, the write word line TWL is set to a low level. At this time, the two NMOS tubes N5 and N6 of the write channel are in the off state, and the write bit lines TBL and TBLB are separated from the intermediate latch structure. At this time, the two NMOS tubes N3 and N4 in the read channel and the two read bit lines BL and BLB can be used to complete the reading of the stored data. Specifically, the 8T-SRAM unit can use any one of BL or BLB alone to realize data reading.

[0056] First, take the data reading through the bit line BL as an example. During the data reading operation, BL is first precharged to a high level, and then WL is set to a high level. At this time, N3 is turned on, and the storage node Q is connected to the bit line BL through N3. In this case, assuming that the original data stored in the storage node is "1", that is, Q=1, QB=0; then the level state between Q and BL is the same, and BL remains at a high level. On the contrary, if the original data stored in the storage node is "0", that is, Q=0, QB=1; then the level state between Q and BL is different, and a discharge path is formed between the bit line BL and the storage node, and BL drops from a high level to a low level.

[0057] That is, when reading through the bit line BL, if BL maintains a high level, the original stored data read is 1; if BL drops from a high level to a low level, the original stored data read is 0.

[0058] Correspondingly, assuming that data is read through one side of the bit line BL, during the data reading operation, it is necessary to precharge BLB to a high level first, and then set RWL to a high level; at this time, N4 is turned on, and the storage node QB is connected to the bit line BLB through N4. In this case, assuming that the data originally stored in the storage node is "1", that is, Q=1, QB=0; then the level state between QB and BLB is different, and a discharge path is formed between the bit line BLB and the storage node QB, and BLB drops from a high level to a low level. Conversely, if the data originally stored in the storage node is "0", that is, Q=0, QB=1; then the level state between QB and BLB is the same, and BLB still remains at a high level.

[0059] That is, when reading through the bit line BLB, if BLB maintains a high level, the original storage data read is 0; if BL drops from a high level to a low level, the original storage data read is 1.

[0060] (III) Data retention

[0061] When N3~N6 are all turned off, the 8T-SRAM unit is in a data retention state, and the stored data can be latched for a long time.

[0062] It should be particularly emphasized that the XT unit in the signed multiplication circuit provided in this embodiment can be used Figure 1 The 8T-SRAM cell in the embodiment may also adopt other types of SRAM cells with read-write separation and dual read ports. For example, on the basis of 8T-SRAM, by adding components and changing the circuit design, a series of related SRAM cells such as 9T, 10T, etc. can be obtained.

[0063] In the signed multiplication circuit provided in this embodiment, the sign operation unit is composed of three AND gates AND1, AND2, and AND3 and a D flip-flop D1. The circuit connection relationship is: the output end of AND1 is connected to the D end of D1; the positive phase output end Q1 of D1 is connected to one of the input ends of AND2; the negative phase output end Q2 is connected to one of the input ends of AND3. The other output ends of AND2 and AND3 are both connected to the operation enable signal EN. The output end of AND2 is connected to the gate of N4 in each XT unit; the output end of AND3 is connected to the gate of N3 in each XT unit.

[0064] In the sign operation unit, the sign bit of the product of the operand and the weight bit is first calculated through an AND gate AND1. Under the premise that the operand is always positive, if the sign of the weight is positive, the sign of the product is "+", and the output of AND1 is 1. Conversely, if the sign of the weight is negative, the sign of the product is "-", and the output of AND1 is 0. The result (1 or 0) obtained by the sign bit operation is output to the D flip-flop. After the triggering of the signal, the D flip-flop generates two opposite signals QB1 and QB2. These two signals are then ANDed with the operation enable signal EN: if both are 1, the voltage pulse is merged, and if there is 0, no signal is generated; and finally the control signals WL and RWL for controlling the read port switch of each XT unit are obtained. Since the two outputs of the D flip-flop are inverted, it means that during the operation process, WL and RWL are also inverted, so only one of the read bit lines BL and BLB can form a discharge path with the middle XT unit at a time, so the sign of the final product result can be obtained according to which side the bit line forming the discharge path is located.

[0065] Furthermore, in the symbol operation unit of the present embodiment, the pulse widths of WL and RWL depend on the pulse width of the enable signal EN (i.e., the value bit N1 of the operand encoded by the pulse width). When the pulse widths of WL and RWL are different, the time for the read bit line BL or BLB to discharge the XT unit will be different; thus, the final multiplication result can be reflected by the difference in the voltage accumulation discharge process.

[0066] Specifically, in the embodiment designed as Figure 1 In the signed multiplication circuit shown, when implementing the multiplication operation between the positive operand and the signed weight, the two input terminals of AND1 are used to input the sign bits S1 and S2 of the operand and the weight respectively. The encoding rule of the sign bit of the operand and the weight is: when S1=1, it means that the sign bit of the operand is "+"; when S1=0, it means that the sign bit of the operand is "-". When S2=1, it means that the sign bit of the weight is "+"; when S2=0, it means that the sign bit of the operand is "-". In particular, considering that the circuit method of this embodiment is only used to perform the multiplication operation between the positive operand and the signed weight, therefore, in practical applications, S1 is often a high level.

[0067] The enable signal EN is used to input the numerical bit N1 of the operand using pulse width coding. Specifically, the unit pulse width △t is defined to represent the numerical value of the operand as 1, and the multiple of the high-level pulse width of the S1 input relative to the unit pulse width △t is the decimal value corresponding to the operand. For example, when the pulse width of the input enable signal is 3△t, it means that the numerical value of the operand is 3. The binary code of the corresponding operand is "111"; among them, the first bit is the sign bit, indicating that the operand is a positive number, and the last two bits are the numerical bits, indicating that the numerical value of the operand is 3.

[0068] In the signed multiplication circuit of this embodiment, the numerical bit of the weight is pre-stored in the storage node of the enabled XT unit before the operation, and the storage node in each XT unit is used to pre-store one of the numerical bits of the weight. Specifically, when Q=1, QB=0 indicates that the numerical bit of the weight is "1"; when Q=0, QB=1 indicates that the numerical bit of the weight is "0".

[0069] In the signed multiplication circuit provided in this embodiment, the product result is reflected in the level change of the read bit lines BL and BLB. Specifically, the read bit lines BL and BLB are precharged to a high level before the operation. After the operation, if the level of BL drops, it means that the product result is a negative number, and the ratio of the level drop of BL relative to the unit drop △V is the value of the product result. For example, if the level amplitude on the BL side is reduced to 25△V, it means that the final product result is "-25". If the level of BLB drops, it means that the product result is a positive number, and the ratio of the level drop of BLB relative to the unit drop △V is the value of the product result. For example, if the level amplitude on the BL side is reduced to 18△V, it means that the final product result is "+18".

[0070] It should be noted that since the circuit designed in this embodiment is a circuit used to process multiplication operations between signed numbers, and the actual application scenario does not involve the situation where the operand or weight is 0, the circuit does not include the logical operation function of multiplication of any number and 0.

[0071] Figure 1 The provided circuit can be used to perform multiplication operations between a multi-bit positive operand and a 2-bit weight, wherein the 2-bit weight includes a one-bit sign bit and a one-bit value bit, that is, the circuit can support multiplication operations between a positive operand and +1, -1. On this basis, when a multiplication operation between an operand and a multi-bit (>2) weight is required, multiple XT units are required, and the value of one of the value bits of the weight is stored in each XT unit. Figure 2 As shown, in the numerical operation unit, when multiple XT units are included, each XT unit shares the same set of read bit lines BL and BLB. In specific applications, the values ​​of each bit of the sign bit of the multi-bit weight are pre-stored in different XT units, and the output ends of AND2 and AND3 in the sign operation unit are simultaneously connected to the read port of each XT unit. In particular, in order to realize different bit weights (different bits for pre-storing the numerical part of the multi-bit weight) on each XT unit, the width-to-length ratios of the MOS tubes on the read channels of each XT unit used in this embodiment are different.

[0072] For example, when you need to perform a multiplication operation of a (N+1)-bit signed weight and a signed operand, you need to enable Figure 2 N XT units in the signed multiplication circuit shown; each XT unit is used to pre-store the values ​​of different weight bits on the numerical bits of the multi-bit weight; in the subsequent operation, the sign operation unit completes the sign bit operation of the operand and the weight, and generates the corresponding control signal WL or RWL according to the sign product result and the value of the operand, and then outputs it to all XT units at the same time, and then completes the multiplication between the operand and each bit in the weight on each XT unit. Since the width-to-length ratio of different XTs is different, when each XT unit transfers charge between BL and BLB according to the product result, the drop amplitude of the bit line voltage of BL or BLB is different, that is, "weighting" of different bits is achieved. In addition, since all XT units are connected to the same set of read bit lines BL and BLB, the multiplication results of each bit of the operand and the weight will be accumulated on the same bit line, that is, the multiplication of multi-bit operands and multi-bit weights is achieved.

[0073] Example 2

[0074] Based on the solution of embodiment 1, this embodiment further provides a column-level MAC circuit, such as Figure 3 As shown, the column-level MAC circuit is composed of a plurality of signed multiplication circuits arranged in columns as in Embodiment 1. In the column-level MAC circuit, all signed multiplication circuits share the same set of read bit lines BL and BLB.

[0075] exist Figure 3 In the column-level MAC circuit shown, each signed multiplication circuit is used to synchronously perform the tasks of each multiplication operation in the MAC operation. In actual application, the operation result of each signed multiplication circuit in the column-level MAC circuit may be a positive number or a negative number. Among them, all positive terms of the multiplication operation included in the MAC operation are accumulated on the bit line BLB; all negative terms of the multiplication operation included in the MAC operation are accumulated on the bit line BL. Finally, the sum of all positive terms in the MAC operation result is reflected on the read bit line BLB, and the sum of all negative terms included in the MAC operation result is reflected on the read bit line BL.

[0076] In order to quantize and output the MAC operation results in the column-level MAC circuit, such as Figure 4 As shown, the column-level MAC circuit also includes or is connected to a quantization circuit. The quantization circuit is mainly composed of two analog-to-digital converters ADC1 and ADC2, and a signed adder ADD. ADC1 and ADC2 are used to quantize the operation results on BL and BLB respectively. The outputs of ADC1 and ADC2 serve as two inputs of ADD; the output of ADD is the final result of the MAC operation.

[0077] For example, suppose a multiplication-accumulation operation is: (+2)×(+2)+(+1)×(-8)+(+2)×(-3)=(-10). Then, in the operation stage, three signed multiplication circuits in the column-level MAC circuit need to be called to enable. In the first signed multiplication circuit, (+2)×(+2) is executed; in the second signed multiplication circuit, (+1)×(-8) is executed; in the third signed multiplication circuit, (+2)×(-3) is executed. After the operation is completed, the product of the first signed multiplication circuit is +4, which causes the read bit line BLB to discharge, and the voltage drops by 4△V; the product of the second signed multiplication circuit is -8, which causes the read bit line BL to discharge, and the voltage drops by 8△V; the product of the third signed multiplication circuit is -6, which causes the read bit line BL to discharge, and the voltage drops by 6△V.

[0078] In the quantization stage, ADC1 quantizes the bit line BL, and the sum of the negative terms of the MAC is (-14); ADC2 quantizes the bit line BLB, and the sum of the positive terms of the MAC is (+4). ADD adds the digital values ​​output by ADC1 and ADC2, and the final MAC result is (-10).

[0079] Example 3

[0080] Based on the solutions in Embodiment 1 and Embodiment 2, this embodiment further provides a MAC operation maximum value finding circuit, which is used to find the operation task with the largest absolute value of the operation result among multiple signed MAC operation tasks. In particular, the circuit provided in this embodiment innovates the processing idea of ​​maximum value search, and can directly determine the operation with the largest absolute value of the MAC result without quantifying the final results of different MAC operations.

[0081] Specifically, Figure 5 As shown, the MAC operation maximum value search circuit includes: a storage and calculation array, a maximum value search module and a peripheral circuit.

[0082] The storage and calculation array is composed of a plurality of column-level MAC circuits as those in Embodiment 2 arranged in rows; each column in the storage and calculation array is used to perform an independent MAC calculation task.

[0083] The maximum value search module includes a sense amplifier array and a decoder. The sense amplifier array is composed of a plurality of voltage latch sense amplifiers SA and a plurality of XOR gates XOR corresponding to the number of columns of the storage and calculation array; the two input ports of each sense amplifier are respectively connected to the bit lines BL and BLB of one of the column-level MAC circuits in the storage and calculation array; the output terminals OUT and OUTB of each sense amplifier are connected to the two input terminals of XOR, and the output terminal of XOR is connected to the input terminal of the decoder. The decoder is used to latch the corresponding column that first generates the output signal in each XOR gate, and then generate a corresponding column code and output it.

[0084] Specifically, in a typical solution, the voltage latch type sensitive amplifier required in the MAC operation maximum value finding circuit can be a sensitive amplifier composed of 6 PMOS tubes PM1 to PM6 and 2 NMOS tubes NM1 and NM2. Figure 6 As shown, the circuit connection relationship of the sense amplifier is:

[0085] The sources of PM1~PM4 are connected to the power supply; the drains of PM1, PM2, NM1, and PM5 are connected to the gates of PM3 and NM2 and serve as the positive output node OUT. The drains of PM3, PM4, PM6, and NM2 are connected to the gates of PM2 and NM1 and serve as the negative output node OUTB. The sources of NM1 and NM2 are connected to the drain of NM3; the gates of PM5, PM6, and NM3 are connected to the enable signal SAE; the gates of PM1 and PM4 are connected to the pre-charge enable signal Pre; the source of PM5 is connected to BL; the source of PM6 is connected to BLB; the source of NM3 is grounded.

[0086] Figure 6 The working logic of SA shown in the figure is as follows: First, both bit lines are precharged to a high level, and the precharge enable signal PRE is set to a low level, and the two opposite nodes OUT and OUTB of SA are initialized to a high level. Secondly, the word line of the SRAM storage cell is turned on, and the bit line on the side where the storage node is "0" begins to discharge. Then the enable signal SAE is set to a low level, PM5 and PM6 are turned on, the bit line is connected to the node of SA, and the potential of the SA node on the same side is pulled down. When the voltage difference is greater than the offset voltage of SA, SAE is set to a high level, SA is turned on, and PM5 and PM6 are turned off to disconnect the bit line from the internal node of SA. Finally, SA internal nodes OUT and OUTB discharge to ground through NM1 and NM2 to NM3 respectively, and one of the internal nodes with lower potential discharges faster and is quickly pulled down to a low level. According to the cross-coupling feedback mechanism, the other internal node rises to a high level, completing the amplification of the bit line voltage difference and realizing the reading of the storage data or operation results in the connected XT unit.

[0087] In the solution of this embodiment, the MAC operation maximum value search circuit provided is a CIM circuit designed based on an SRAM circuit; therefore, the peripheral circuit in this solution refers to other necessary circuits related to the execution of data storage functions and the execution of signed multiplication and MAC operations and other logical operation functions, such as pre-charging circuits, read-write drive circuits, input drive circuits, module switching modules, global cache modules, and so on.

[0088] The circuit principle of the MAC operation maximum value finding circuit provided in this embodiment is:

[0089] First, the logic operation body of the MAC operation maximum value search circuit of this embodiment is composed of multiple columns of column-level MAC circuits. Combining the contents of Embodiment 1 and Embodiment 2, it can be seen that the MAC result calculated by each column-level MAC circuit will reflect the sum of all positive terms and the sum of negative terms contained therein on two read bit lines BLB and BL respectively. Therefore, for this new type of column-level MAC circuit, the absolute value of the final MAC result is essentially the difference between the absolute values ​​of the results on BL and BLB.

[0090] In view of this situation, the MAC operation maximum value finding circuit of this embodiment adds a maximum value finding module below the read bit line of the storage and calculation array. In the maximum value finding module, the read bit lines BL and BLB of each column-level MAC circuit are connected to the two input ends of one of the sense amplifiers. Therefore, the sense amplifier and the XOR gate can compare the absolute value of the positive value and the absolute value of the negative value of the MAC operation result of each column.

[0091] Specifically, when the result of the multi-bit multiplication and accumulation with a sign bit is accumulated on BL and BLB, BL and BLB are connected to the two input ports of the voltage latch type sensitive amplifier SA respectively, so that the voltage on BL representing a positive value and BLB representing a negative value is input into SA for comparison, and SA with a larger input difference between the two outputs the results OUT and OUTB at the fastest speed. At this time, because the initial states of OUT and OUTB of SA have been set to 1, due to the difference in voltage on BL and BLB, if the voltage difference between BL and BLB is positive, OUT remains unchanged at 1, and if the voltage difference is negative, OUT is discharged to 0. Then the OUT signal and OUTB signal are connected to the two input terminals of the XOR gate respectively, and the output of the XOR gate will change according to the changes of the OUT signal and OUTB signal.

[0092] Based on the basic principle of the circuit composed of the sense amplifier and the XOR gate, it can be known that if the positive and negative parts of a MAC operation are close, the SA needs to spend more time to compare the difference between the two. On the contrary, if the positive and negative parts of a MAC operation are very different, the SA only needs less time to compare the difference between the two.

[0093] This shows that the difference between the positive and negative parts of the MAC operation can be determined by using the order of the output signals generated by each SA, and the difference between the positive and negative parts of the MAC operation is the absolute value of the operation result in the MAC operation. That is, if in the circuit of this embodiment, the SA under a column-level MAC circuit first outputs the output signal representing the comparison result, it means that the absolute value of the MAC operation result executed in the column-level MAC circuit is the largest.

[0094] In order to decode the maximum value results obtained by different sense amplifier arrays, this embodiment adds a decoder in the maximum value search module, and the decoder includes a plurality of input ports corresponding to the number of sense amplifiers (the number of columns of the storage and calculation array) and an output port. Each input port of the decoder is connected to the output end of each sense amplifier, and the output port of the decoder is used to output the column code representing the column where the maximum value of the MAC operation result is located.

[0095] For example, assuming that each storage and calculation array includes 32 columns, the input ports of the decoder can be defined as A0, A1, A2...A31; at this time, the outputs of the sensitive amplifiers of each column are encoded in sequence to obtain a 32-bit binary code A0~A31.

[0096] In the initial state, A0~A31 are all 0. When the output of SA of a certain input port switches from 0 to 1 first, it means that the circuit has recognized that the absolute value of the MAC result of this column is the largest. At this time, the decoder latches the binary code composed of all input signals at the current moment. Only one bit in the binary code is 1, and the rest are 0.

[0097] The decoder provided in this embodiment can convert the binary code into a sequence number of a bit having a value of 1 in the entire binary code according to a preset operation logic (such as table lookup).

[0098] Assuming that the column numbers of the storage and calculation array are encoded starting from 0, when the decoder uses the table lookup method to decode and output, the mapping relationship it uses is as follows:

[0099]

[0100] Example 4

[0101] Based on the foregoing embodiments, this embodiment further provides a neural network accelerator chip, which is used to perform neural network computing tasks with a self-attention mechanism. The chip adopts a MAC computing maximum value finding circuit as in Embodiment 3.

[0102] Considering that during the execution of a neural network computing task with a self-attention mechanism, it is necessary to locate the operation with the largest absolute value of the results among multiple MAC operations, and the MAC operation maximum value finding circuit used in the chip provided by this embodiment can complete this task more quickly and with low power consumption. Compared with the existing logic circuits that perform similar tasks, the circuit used in the chip of this embodiment occupies less on-chip area, and can realize maximum value search without relying on quantization results, and the computing efficiency is significantly higher; therefore, the solution of this embodiment helps to improve the performance of the neural network accelerator.

[0103] Performance Testing

[0104] In order to verify the performance of the signed multiplication circuit, column-level MAC circuit and MAC operation maximum value finding circuit provided by the present invention, the technicians used Figure 2 The 8T-SRAM unit shown is a numerical operation unit, and then in the simulation software Figure 5 The circuit functionality of the CIM circuit shown is tested.

[0105] 1. Multiplication

[0106] In this experiment, we first select six rows of 8T cells in one column of the circuit to form Figure 7 The two signed multiplication circuits shown in FIG. 1 and FIG. 2 respectively implement a multiplication operation between a group of signed operands and signed weights. Since the circuit includes 3 rows of 8T units, the signed multiplication circuit can implement a 4-bit×4-bit operation.

[0107] Specifically, two signed multiplication circuits are used to perform the calculation process of + 6 (Input) × -5 (Weight) and + 7 (Input) × +3 (Weight) respectively. First, the weights -5 and +6 are stored in QB and Q respectively. And BL and BLB are pre-filled. Then the product of the sign bit is calculated in the two sign operation units and a signal is generated, and the sign of the first calculation result is negative and the sign of the second calculation result is positive. Therefore, the sign operation unit controls the RWL of the first 8T unit to be low, and the input pulse with weight reaches WL; controls the WL of the second 8T unit to be low, and RWL inputs a pulse. This will control BL to discharge the first 8T unit and BLB to discharge the second 8T unit.

[0108] Finally, according to the waveform of the discharge result, in the first multiplication operation, the discharge amount of BL is 30ΔV, and in the second multiplication operation, the discharge amount of BLB is 21ΔV. That is, the result of the first multiplication operation is (+ 6) × (-5) = (-30), and the result of the second operation is (+ 7) × (+3) = (+21).

[0109] 2. Find the maximum absolute value of the product

[0110] In this experiment, the 8T unit with six rows and three columns is selected as follows Figure 8 The MAC operation absolute value maximum finding circuit is shown, wherein each column completes a MAC operation respectively.

[0111] Specifically, in this experiment, the operands selected by the three-column circuits in the two multiplication tasks of the MAC operation are both +6 and +3; the operands are multiplied by the weights of the three columns respectively.

[0112] Among them, the weights of the two multiplication operations in column 0 are -2 and +7. During the operation, the numerical bit 2 of the weight of the first multiplication will be stored in QB according to the binary code. After the sign operation, WL will be turned on. Because BL and BLB are pre-charged with VDD, BL will discharge 12ΔV to Q point. RWL is at a low level at this time and remains off. The numerical bit 7 of the weight of the second multiplication will be stored in Q according to the binary code. After the sign operation, RWL will be turned on, and BLB will discharge 21ΔV to QB. Finally, the MAC operation result of column 0 includes both positive and negative parts. The positive part is reflected on BLB, which is: VDD-21ΔV, and the negative part is reflected on BL, which is: VDD-12ΔV.

[0113] Correspondingly, the weights of the first column are +4 and +6, and the numerical values ​​of the weights of the first and second multiplications, bits 4 and 6, are stored in Q according to the binary code. After the sign operation, RWL is turned on and WL is set to a low level. BLB discharges 42ΔV to QB, so the MAC operation result of the first column only includes the positive part, which is reflected on BLB as: VDD-42ΔV. The bit line voltage of BL is VDD, indicating that the product result does not include negative components.

[0114] The weights of the second column are -6 and +1. The value bit 6 of the weight of the first multiplication is stored in QB according to the binary code. After the sign bit operation, WL is turned on and RWL is set to a low level. BL discharges 36ΔV to Q. The value bit 1 of the weight of the second multiplication is stored in Q according to the binary code. After the sign operation, RWL is turned on, WL is set to a low level, and BLB discharges 3ΔV to QB. Therefore, the MAC operation result of the second column includes both positive and negative parts. Among them, the positive part is reflected on the bit line BLB, which is: VDD-3ΔV, and the negative part is reflected on the bit line BL, which is: VDD-36ΔV.

[0115] Finally, after the MAC operation results of the three columns are processed by the maximum value search module, it can be seen from the level states of the different bit lines of the three columns that the SA corresponding to the column with the largest difference between BL and BLB (i.e., column 1) will flip the fastest, and the XOR gate of this column will output 1 the fastest. Therefore, the decoder can immediately obtain a string of binary codes 010 and convert it into the number of the column where the absolute maximum value of the MAC result is located, i.e., column 1.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A signed multiplication circuit for performing a multiplication operation between a positive operand and a signed weight, characterized in that: It includes a numerical operation unit and a symbol operation unit; the numerical operation unit includes at least one XT unit; the XT unit adopts a read-write separation SRAM unit with a dual read port; the read port of the XT unit is composed of two NMOS tubes N3 and N4; N3 is used as a transmission tube between the storage node Q and the read bit line BL; N4 is used as a transmission tube between the storage node QB and the read bit line BLB; The symbol operation unit is composed of three AND gates AND1, AND2, and AND3 and a D flip-flop D1; the circuit connection relationship is: the output end of AND1 is connected to the D end of D1; the positive phase output end Q1 of D1 is connected to one of the input ends of AND2; the negative phase output end Q2 is connected to one of the input ends of AND3; the other output ends of AND2 and AND3 are both connected to the operation enable signal EN; the output end of AND2 is connected to the gate of N4 in each XT unit; the output end of AND3 is connected to the gate of N3 in each XT unit; Among them, the two input ends of AND1 are used to input the sign bits S1 and S2 of the operand and the weight respectively; the enable signal EN is used to input the numerical bit N1 of the operand using pulse width encoding; the storage node in each XT unit is used to pre-store one of the numerical bits of the weight; the product result is reflected in the level change of the read bit lines BL and BLB.

2. The signed multiplication circuit according to claim 1, wherein: The encoding rules for the sign bits of operands and weights are: When S1=1, it indicates that the operand's sign bit is "+"; when S1=0, it indicates that the operand's sign bit is "-"; When S2=1, it means the sign bit of the weight is "+"; when S2=0, it means the sign bit of the operand is "-"; The encoding rules for the numerical bits of operands and weights are: Define the unit pulse width △t to represent the value of the operand as 1, then the multiple of the high level pulse width of S1 input relative to the unit pulse width △t is the decimal value of the operand; When Q=1, QB=0, it means that the value of the weight is "1"; when Q=0, QB=1, it means that the value of the weight is "0".

3. The signed multiplication circuit as claimed in claim 2, characterized in that: Before the operation, the read bit lines BL and BLB are precharged to a high level. After the operation, if the level of BL drops, it means that the product result is a negative number, and the ratio of the level drop of BL to the unit drop △V is the value of the product result; if the level of BLB drops, it means that the product result is a positive number, and the ratio of the level drop of BLB to the unit drop △V is the value of the product result.

4. The signed multiplication circuit according to claim 1, wherein: In the numerical operation unit, when a plurality of XT units are included, each XT unit shares the same set of read bit lines BL and BLB; When it is necessary to complete the multiplication operation of a (N+1)-bit signed weight and a signed operand, N XT units need to be enabled; each XT unit is used to pre-store the values ​​of different weight bits on the numerical bits of the multi-bit weight; the bit weights of different XT units are represented by the width-to-length ratio of the left and right units of the 8T unit.

5. The signed multiplication circuit according to claim 1, wherein: The XT unit in the numerical operation unit adopts a read-write separated 8T-SRAM unit; the 8T-SRAM unit is composed of two PMOS tubes P1 and P2 and six NMOS tubes N1 to N6; the circuit connection relationship is: P1, P2, N1, and N2 are anti-phase cross-coupled to form a latch structure including storage nodes Q and QB; N3 and N4 form a read channel; N3 serves as a transmission tube between the storage node Q and the read bit line BL; N4 serves as a transmission tube between the storage node QB and the read bit line BLB; N5 and N6 form a write channel; N5 serves as a transmission tube between the storage node Q and the write bit line TBL; N6 serves as a transmission tube between the storage node QB and the write bit line TBLB; the gates of N5 and N6 are connected to the write word line TWL.

6. A column-level MAC circuit, characterized in that: It is composed of a plurality of signed multiplication circuits as claimed in any one of claims 1 to 5 arranged in columns; each signed multiplication circuit shares the same set of read bit lines BL and BLB; In the column-level MAC circuit, each signed multiplication circuit is used to synchronously execute the tasks of each multiplication operation in the MAC operation; then the sum of all positive terms included in the MAC operation is reflected on the read bit line BLB, and the sum of all negative terms included in the MAC operation is reflected on the read bit line BL.

7. The column-level MAC circuit according to claim 6, wherein: It also includes or is connected to a quantization circuit; the quantization circuit includes two analog-to-digital converters ADC1 and ADC2, and a signed adder ADD; ADC1 and ADC2 are used to quantize the operation results on BL and BLB respectively; the outputs of ADC1 and ADC2 are used as two inputs of ADD, and the output of ADD is the final result of the MAC operation.

8. A MAC operation maximum value finding circuit, characterized in that: It is used to find the operation task with the largest absolute value of the operation result among multiple signed MAC operation tasks; the MAC operation maximum value finding circuit includes: A storage and calculation array, which is composed of a plurality of column-level MAC circuits as claimed in claim 7 arranged in rows; each column in the storage and calculation array is used to perform an independent MAC operation task; A maximum value search module, comprising a sense amplifier array and a decoder; the sense amplifier array is composed of a plurality of voltage latched sense amplifiers and XOR gates corresponding to the number of columns of the storage and calculation array; the two input ports of each sense amplifier are respectively connected to the bit lines BL and BLB of one of the column-level MAC circuits in the storage and calculation array; the output terminals OUT and OUTB of each sense amplifier are connected to the two input terminals of the XOR, and the output terminal of the XOR is connected to the input terminal of the decoder; the decoder is used to latch the corresponding column that first generates an output signal in each XOR gate, and then generate a corresponding column code and output it; and Peripheral circuits, which include other necessary circuits related to performing data storage and logic operations.

9. The MAC operation maximum value finding circuit as claimed in claim 8, characterized in that: The sensitive amplifier is composed of 6 PMOS tubes PM1 to PM6 and 2 NMOS tubes NM1 and NM2, and the circuit connection relationship is: The sources of PM1~PM4 are connected to the power supply; the drains of PM1, PM2, NM1, and PM5 are connected to the gates of PM3 and NM2 and serve as the positive output node OUT; the drains of PM3, PM4, PM6, and NM2 are connected to the gates of PM2 and NM1 and serve as the negative output node OUTB; the sources of NM1 and NM2 are connected to the drain of NM3; the gates of PM5, PM6, and NM3 are connected to the enable signal SAE; the gates of PM1 and PM4 are connected to the pre-charge enable signal Pre; the source of PM5 is connected to BL; the source of PM6 is connected to BLB; the source of NM3 is grounded.

10. A neural network accelerator chip for executing neural network computing tasks with a self-attention mechanism, characterized in that: It adopts the MAC operation maximum value finding circuit as claimed in claim 9.

Images