Floating-point multiplication and accumulation fast calculation circuit and chip based on SRAM

Abstract

The present invention belongs to the field of integrated circuit technology, and specifically relates to a floating-point multiplication and accumulation fast operation circuit and its chip based on SRAM. The circuit is designed based on an SRAM array and its peripheral circuits, wherein the SRAM array is divided into an exponent sum array, a weighted exponent array and a weighted mantissa array by column. Based on the divided SRAM array, the floating-point multiplication and accumulation fast operation circuit also includes: an exponent input module, a mantissa input module, an adder array, a maximum value search module, a subtraction counter, a shift register, an adder tree and a normalization module. The present invention adopts a new high-bandwidth asynchronous exponent normalization and instruction parallel sorting mantissa alignment floating-point calculation process, which can find the maximum value in parallel while adding the exponent, and replace the subtraction and shift in the mantissa alignment by a time period search method, thereby realizing MAC in-memory calculation of floating-point data with lower time, area and power consumption.

Description

Technical Field

[0001] The invention belongs to the technical field of integrated circuits, and in particular relates to a floating-point multiplication-accumulation fast operation circuit based on SRAM and a CIM chip using the circuit. Background Art

[0002] In recent years, with breakthroughs in key technologies such as big data and artificial intelligence (AI), emerging intelligent applications, exemplified by edge computing and smart living, have emerged in a rapidly evolving era. These emerging intelligent applications often require frequent memory access when processing events. However, the von Neumann architecture, the most commonly used data processing architecture, is implemented by separating memory and computational units. Initially, computational units and storage units developed in parallel, meaning that the storage speed of memory and the computing speed of the arithmetic unit remained consistent. However, as semiconductor technology continued to advance in line with Moore's Law, computational units advanced in speed, while storage continued to increase in integration. This resulted in a lag between the storage speed of memory and the computing speed of the arithmetic unit, commonly known as the "memory wall."

[0003] Research has shown that the time and power consumption required to access data are far greater than those required for computation, and this difference will become more pronounced as technology continues to advance. Achieving efficient, low-overhead computing on resource-limited devices is a key future development direction. To address this, researchers have proposed a memory-based computing-in-memory (CIM) architecture. This new computer architecture directly utilizes memory to perform logical operations, eliminating the need for data transfer between memory and the processor. This significantly improves data processing efficiency and reduces device operating power consumption.

[0004] With the development of deep learning algorithms, the scale and complexity of neural networks continue to increase. These large-scale neural network models are now widely used in computer vision, natural language processing, and recommendation systems. Their diverse application scenarios demonstrate their unique importance. Traditional integer neural networks are inherently disadvantaged in large-scale model calculations due to their limited data representation range and low computational accuracy. Floating-point neural networks effectively address this issue. With their wider numerical representation range and computational accuracy, floating-point neural networks are more suitable for use in large-scale neural network models.

[0005] Data processing based on deep learning algorithms involves a large number of multiply-accumulate (MAC) tasks. Existing SRAM-based CIM circuits with MAC functionality are highly efficient for integer operations. However, when performing floating-point operations, traditional CIM circuits require multiple sequential steps, including exponent addition, mantissa multiplication, exponent alignment, mantissa shifting, and data synthesis. This often requires multiple cycles to obtain the final multiply-accumulate result, significantly reducing the efficiency of the circuit's multiply-accumulate operations on floating-point data. Summary of the Invention

[0006] In order to solve the problem of low efficiency and slow speed of existing CIM circuits in processing floating-point data multiplication and accumulation operations, the present invention provides an SRAM-based floating-point multiplication and accumulation fast operation circuit and a CIM chip using the circuit.

[0007] The technical solution provided by the present invention is:

[0008] A fast SRAM-based floating-point multiplication-accumulation circuit is designed based on an SRAM array and its peripheral circuits, and is used to implement multiplication-accumulation operations between multiple groups of multi-bit floating-point operands and multi-bit floating-point weights. In the present invention, the SRAM array in the fast floating-point multiplication-accumulation circuit is divided into three parts by column: an exponent sum array, a weight exponent array, and a weight mantissa array.

[0009] Based on the divided SRAM array, the floating-point multiplication and accumulation fast operation circuit also includes: an exponent input module, a mantissa input module, an adder array, a maximum value search module, a subtraction counter, a shift register, an adder tree and a normalization module.

[0010] Among them, each row of the weight exponent array is used to pre-store the exponent part of the weight bit by bit. The exponent input module is used to input the exponent part of the operand to each row in the adder array. The adder array is used to read the stored value of the weight exponent array and calculate the sum of the exponents of the weight and the operand. Each row in the weight mantissa array is used to pre-store the mantissa part of the weight bit by bit. The mantissa input module is used to input the mantissa part of the operand to each row in the weight mantissa array. Each row in the weight mantissa array is used to pre-store the mantissa part of the weight bit by bit, and uses its own logical operation function to perform multiplication operation on the mantissa part of the weight and the operand to obtain the mantissa product.

[0011] The maximum value finding module first obtains the exponential sum calculation results of each row output by the adder array and determines the maximum exponent while writing the exponential sum back to the exponential sum array bit by bit. It then uses a subtraction counter and the exponential sum array to determine the number of bits difference between the exponential sum calculated in each row and the maximum exponent. A shift register then shifts the mantissa product calculated in the corresponding row by this number of bits.

[0012] The adder tree is used to add the mantissa products of each row after the shift is completed to obtain the mantissa sum; finally, the normalization module generates the corresponding multiplication and accumulation operation result according to the maximum exponent and the mantissa sum.

[0013] The SRAM-based floating-point multiplication and accumulation fast operation circuit provided by the present invention adopts a pipeline strategy to complete the multiplication and accumulation operation task of floating-point numbers in practical applications. The process is as follows:

[0014] S1: Initial stage:

[0015] In the first cycle, the exponential input module, the weight exponential array and the adder array are used to complete the input and addition of the exponential parts of each operand and weight in the first round of operation tasks to obtain the exponential sum.

[0016] The maximum value search module reads the exponents and calculation results of each row in the adder array bit by bit in order from low to high.

[0017] S2: Operation stage:

[0018] In the second cycle, the mantissa input module and the weight mantissa array are used to complete the input and multiplication of the mantissa parts of each operand and weight in the first round of calculation tasks to obtain each mantissa product.

[0019] The maximum value finding module writes the calculated result of the read exponent sum back to the exponent sum array and determines the maximum exponent at the same time. The maximum exponent is then decremented by a subtraction counter and compared with each exponent sum in the exponent sum array. After each comparison, the shift register shifts the mantissa product of the exponent sum that is smaller than the maximum exponent.

[0020] The adder tree then adds the mantissa products of the shifted rows to obtain the mantissa sum. Finally, the normalization module generates the corresponding multiply-accumulate result based on the determined maximum exponent and mantissa sum.

[0021] At the same time, in the second cycle, the adder array is also used to complete the input and addition of the exponential parts of each operand and weight in the second round of calculation tasks, and the maximum value search module reads the calculation results of the exponential sum of each row in the adder array in order from low to high.

[0022] S3: Cycle phase:

[0023] Following the same logic, in each subsequent cycle, the multiplication and accumulation results of the previous round are calculated and output, and the addition and caching of the exponential part of the shift round operation are completed;

[0024] Similarly, in the Nth cycle, the processing and output of the N-1th round of multiplication and accumulation operations can be achieved.

[0025] As a further improvement of the present invention, the maximum value search module includes a plurality of ping-pong units and a plurality of comparison units corresponding to the number of rows of the SRAM array; each ping-pong unit includes register 1 and register 2.

[0026] In the previous cycle, the ping-pong unit is used to read the exponential sums calculated by the adder array in order from low to high and store them in register one; and write each bit of data in register one to register two bit by bit, and the data written into register two is restored to the exponential sum arranged from high to low.

[0027] In the next cycle, the ping-pong unit writes the data in register 2 back to the corresponding row in the index and array, and simultaneously determines the maximum exponent during the write-back process. At the same time, the ping-pong unit also performs the index and read in the next round of calculation and updates register 1.

[0028] The comparison unit is used to compare the exponents of each row after writing back with the operation result to determine the maximum exponent.

[0029] As a further improvement of the present invention, the maximum value finding module determines the maximum exponent in the process of writing back the sum of the exponents of each row as follows:

[0030] (1) The exponential sum calculated by the adder array is written bit by bit from register 2 to the exponential sum array in descending order.

[0031] (2) When writing each bit of the exponent sum, determine the current value and make the following decisions:

[0032] A. If the values ​​of the current positions of the sum of the exponents in different rows include both 1 and 0, the numbers whose current positions are 1 are retained and the numbers whose current positions are 0 are removed.

[0033] B. If the values ​​of the current bits of the sums of the exponents in different rows are both 1 or 0, then the numbers on all rows are retained.

[0034] (3) If only one number remains or the lowest bit of the sum of the exponents has been written, the number that remains is the maximum exponent.

[0035] As a further improvement of the present invention, the exponent sum array, the subtraction counter, and the shift register implement an operation strategy of shifting the mantissa product according to the maximum exponent output by the maximum value finding module as follows:

[0036] The subtraction counter decrements the maximum exponent by 1 in each cycle and sends it to the exponent sum array.

[0037] The index and array matches the maximum index at the current moment with the index and index stored in itself in each cycle. If the match is successful, a corresponding pairing flag is generated and sent to the shift register.

[0038] The shift register is used to shift the exponent and the corresponding mantissa product of each row in each cycle, and stop the shift operation of the mantissa product of the corresponding row when any pairing flag is received in each cycle.

[0039] When the preset maximum number of cycles is reached, the shifting of all mantissa products is completed.

[0040] As a further improvement of the present invention, the weight mantissa array is composed of multiple 8T-SRAM units; each 8T-SRAM unit includes two PMOS transistors P1-P2 and six NMOS transistors N1-N6. The circuit connection relationship is:

[0041] P1, P2, N1, and N2 are anti-phase cross-coupled to form a set of storage nodes Q and QB. N3 acts as a transmission transistor between Q and bit line BL, and N4 acts as a transmission transistor between QB and bit line BLB. The gates of N3 and N4 are connected to word line W. The gate of N6 is connected to QB, the source of N6 is grounded, and the drain of N6 is connected to the drain of N5. The gate of N5 is connected to the computing word line RWL, and the source of N5 is connected to the computing bit line CL.

[0042] As a further improvement of the present invention, the P1~P2 and N1~N4 parts in the 8T-SRAM unit constitute a 6T storage unit. The 6T storage unit in the 8T-SRAM unit is used to implement the data storage function in the storage mode, and is used to pre-store one of the mantissa parts in the weight under the calculation module.

[0043] The operation logic of the storage unit in conjunction with N5 and N6 to implement multiplication is as follows:

[0044] (1) The tail part of the weight is pre-stored in the storage unit:

[0045] When Q is high and QB is low, the corresponding bit of the mantissa of the weight is "1"; when Q is low and QB is high, the corresponding bit of the mantissa of the weight is "0".

[0046] (2) Precharge the calculation bit line CL to a high level, and then input the mantissa of the operand through the calculation word line RWL:

[0047] When RWL is high, it indicates that the corresponding bit of the mantissa of the input operand is "0". When RWL is low, it indicates that the corresponding bit of the mantissa of the input operand is "1".

[0048] (3) Output the final calculation result of the mantissa product according to the level change of the calculated bit line CL:

[0049] When CL drops to a low level, it indicates that the product result is "1"; when CL remains high, it indicates that the product result is "0".

[0050] As a further improvement of the present invention, the index and array are composed of multiple 10T-SRAM cells; each 10T-SRAM cell includes two PMOS transistors PM0 to PM1 and eight NMOS transistors NM0 to NM7, and the circuit connection relationship is:

[0051] The sources of PM0 and PM1 are connected to VDD; the drain of PM0 is connected to the gates of PM1, NM3, NM4 and the drains of NM0 and NM2, and serves as the storage node Q; the drain of PM1 is connected to the gates of PM0, NM2, NM5 and the drains of NM1 and NM3, and serves as the inverting storage node QB; the sources of NM2 and NM3 are grounded; the source of NM0 is connected to the bit line BL; the source of NM1 is connected to the bit line BLB; the gates of NM0 and NM1 are connected to the word line WL.

[0052] The drain of NM4 and the source of NM5 are connected to the matching bit line TL; the source of NM4 and the drain of NM6 are connected; the drain of NM5 and the source of NM7 are connected; the source of NM6 and the drain of NM7 are connected to the matching bit line ML; the gate of NM6 is connected to the local word line WLL; the gate of NM7 is connected to the local word line WLR.

[0053] As a further improvement of the present invention, the PM0-PM1 and NM0-NM3 portions of the 10T-SRAM cell form a 6T storage unit. The 6T storage unit in the 10T-SRAM cell is used for data storage in storage mode and for restoring one bit of the exponential sum in each row in calculation mode. The remaining 4T portion forms a matching unit. This matching unit uses one bit of the exponential sum stored in the 6T storage unit as the target number and the corresponding bit of the successively decreasing maximum exponent as the query number, then outputs a query result indicating whether the target number matches the query number.

[0054] The operation logic of the matching unit to achieve data matching is as follows:

[0055] (1) Write back each bit of the sum of the weight and the exponent of the operand to the 6T storage unit in each 10T-SRAM unit to form the target number:

[0056] When Q is high and QB is low, it indicates that the target number is "1"; when Q is low and QB is high, it indicates that the target number is "0".

[0057] (2) Precharge the matching bit lines TL and ML to a high level; then input the encoded query number to the matching unit through the local word lines WLL and WLR:

[0058] When WLL is high and WLR is low, it indicates that the query number is "1"; when WLL is low and WLR is high, it indicates that the query number is "0".

[0059] (3) Ground the matching bit line TL and determine the matching result based on the level change of the matching bit line ML:

[0060] When the query number and the target number are the same, a discharge path is formed between ML and TL, and ML drops to a low level, indicating that the two are matched successfully.

[0061] When the query number and the target number are different, a discharge path cannot be formed between ML and TL, and ML remains at a high level, indicating that the matching between the two has failed.

[0062] The present invention also includes a CIM chip that integrates the aforementioned SRAM-based floating-point multiplication-accumulation fast computation circuit. The CIM chip has a storage mode and a computation mode. In the storage mode, the CIM chip functions as a memory. In the computation mode, the CIM chip is used to implement multiplication-accumulation operations between multiple groups of multi-bit floating-point operands and multi-bit floating-point weights.

[0063] The technical solution provided by the present invention has the following beneficial effects:

[0064] The present invention provides a new circuit architecture and operating logic for a floating-point multiplication and accumulation fast operation circuit. The circuit scheme applies a new high-bandwidth asynchronous exponent normalization and instruction parallel sequencing mantissa alignment floating-point calculation process. The maximum value can be found in parallel while adding exponents, and the subtraction and shifting in mantissa alignment are replaced by a time period search method, thereby realizing MAC in-memory calculation of floating-point data with lower time, area and power consumption.

[0065] The present invention rationally allocates and parallelizes the different steps in the floating-point multiplication and accumulation process through the coordinated cooperation of different modules, thereby completing a single multiplication and accumulation task in at most two cycles. The present invention's solution also rationally allocates the processing timing of different steps in the calculation process, thereby shortening the overall time required for a large number of calculation tasks through a "pipelining" approach, limiting the average time required for a single calculation task to approximately one cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a circuit architecture diagram of the SRAM-based floating-point multiplication and accumulation fast operation circuit provided in Example 1 of the present invention.

[0067] Figure 2 This is a circuit schematic diagram of the index and array parts in Example 1 of the present invention.

[0068] Figure 3 This is a circuit diagram of a 10T-SRAM cell used in the index and array in Example 1 of the present invention.

[0069] Figure 4 4 is a circuit schematic diagram of the cascade connection of the matching units in the index and array in Example 1 of the present invention.

[0070] Figure 5 This is a circuit diagram of the 8T-SRAM unit used in the finger weight mantissa array in Example 1 of the present invention.

[0071] Figure 6 This is a timing distribution diagram of the pipeline strategy for floating-point multiplication-accumulation operations used by the circuit in Example 1 of the present invention.

[0072] Figure 7 This is a flow chart of determining the maximum index by the maximum value finding module in embodiment 1 of the present invention.

[0073] Figure 8 This is a circuit module diagram for implementing index and alignment functions in Example 1 of the present invention.

[0074] Figure 9 This is a flow chart of implementing the mantissa product shift operation in embodiment 1 of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0076] Example 1

[0077] This embodiment provides an SRAM-based floating-point multiplication and accumulation fast operation circuit, which is designed based on an SRAM array and its peripheral circuits and is used to implement multiplication and accumulation operations between multiple groups of multi-bit floating-point operands and multi-bit floating-point weights. The floating-point multiplication and accumulation fast operation circuit in this embodiment has two operating modes: storage and calculation, and controls the functional switching of the circuit through a mode switching module. The peripheral circuits of the SRAM array refer to the related circuits and modules related to using the SRAM array to read, write, and save data.

[0078] like Figure 1 As shown, in order to achieve the calculation function, in the solution of this embodiment, the SRAM array in the floating-point multiplication and accumulation fast calculation circuit is divided into three parts by column, namely the exponent sum array, the weight exponent array, and the weight mantissa array. Among them, the weight exponent array is mainly used to process tasks related to the exponent part of the floating-point number during the calculation process; the weight mantissa array is used to process tasks related to the mantissa part of the floating-point number during the calculation process; and the exponent sum array is used to store the sum of the exponents between the operand and the weight.

[0079] Specifically, in practical applications, for a fully functional floating-point multiplication and accumulation fast arithmetic circuit, the number of columns of SRAM cells in the exponent sum array, weight exponent array, and weight mantissa array can be reasonably divided according to the specific number of bits of the exponent part and mantissa part contained in the floating-point number type to be processed. For example, for a floating-point number in BF16 format, it contains 8 exponent bits and 7 decimal places; considering the overflow in the actual operation process, the number of columns of the exponent sum array, weight exponent array, and weight mantissa array can be set to 8:8:8 respectively; while for a floating-point number in FP32 format, it contains 8 exponent bits and 23 decimal places, the number of columns of the exponent sum array, weight exponent array, and weight mantissa array can be set to 8:8:24 respectively.

[0080] Based on the divided SRAM array, the floating-point multiplication and accumulation fast operation circuit provided in this embodiment also includes: an exponent input module, a mantissa input module, an adder array, a maximum value search module, a subtraction counter, a shift register, an adder tree and a normalization module.

[0081] Among them, each row of the weight exponent array is used to pre-store the exponent part of the weight bit by bit. The exponent input module is used to input the exponent part of the operand to each row in the adder array. The adder array is used to read the stored value of the weight exponent array and calculate the sum of the exponents of the weight and the operand. Each row in the weight mantissa array is used to pre-store the mantissa part of the weight bit by bit. The mantissa input module is used to input the mantissa part of the operand to each row in the weight mantissa array. Each row in the weight mantissa array is used to pre-store the mantissa part of the weight bit by bit, and uses its own logical operation function to perform multiplication operation on the mantissa part of the weight and the operand to obtain the mantissa product.

[0082] In practical applications, the exponents or mantissas of the same operand and weight are input bit by bit into the same row of the array, and one of the bits is stored in the same cell of the array. Different rows are used to store different multi-bit exponents or multi-bit mantissas. On this basis, the maximum number of groups of floating-point multiplication and accumulation operands or weights that can be supported by the floating-point multiplication and accumulation fast operation circuit provided in this embodiment is determined by the number of rows of the SRAM array. For example, in a circuit where the scales of the exponent and array, weight exponent array, and weight mantissa array are 64×8, 64×8, and 64×8, respectively, it can support the multiplication and accumulation operations between a maximum of 64 floating-point operands and 64 floating-point weights.

[0083] In this embodiment, the maximum value search module first obtains the exponential sum calculation results for each row output by the adder array and determines the maximum exponent while writing the exponential sum back bit by bit to the exponential sum array. The subtraction counter and the exponential sum array are then combined to determine the number of bits difference between the exponential sum calculated for each row and the maximum exponent. A shift register then shifts the mantissa products calculated for the corresponding row based on the number of bits difference.

[0084] The adder tree is used to add the mantissa products of each row after the shift is completed to obtain the mantissa sum; finally, the normalization module generates the corresponding multiplication and accumulation operation result according to the maximum exponent and the mantissa sum.

[0085] In detail, such as Figure 2 As shown, the index and array in this embodiment are composed of multiple 10T-SRAM units. Figure 3 As shown, each 10T-SRAM cell includes two PMOS transistors PM0 to PM1 and eight NMOS transistors NM0 to NM7. The circuit connection relationship is:

[0086] The sources of PM0 and PM1 are connected to VDD; the drain of PM0 is connected to the gates of PM1, NM3, and NM4, as well as the drains of NM0 and NM2, and serves as storage node Q; the drain of PM1 is connected to the gates of PM0, NM2, and NM5, as well as the drains of NM1 and NM3, and serves as inverted storage node QB; the sources of NM2 and NM3 are grounded; the source of NM0 is connected to bit line BL; the source of NM1 is connected to bit line BLB; the gates of NM0 and NM1 are connected to word line WL. The drain of NM4 and the source of NM5 are connected to matching bit line TL; the source of NM4 is connected to the drain of NM6; the drain of NM5 is connected to the source of NM7; the source of NM6 and the drain of NM7 are connected to matching bit line ML; the gate of NM6 is connected to local word line WLL; and the gate of NM7 is connected to local word line WLR.

[0087] Combine Figure 3 In this embodiment, the PM0-PM1 and NM0-NM3 portions of each 10T-SRAM cell in the index sum array constitute a 6T storage unit. The 6T storage unit in the 10T-SRAM cell can perform data storage in storage mode and restore one bit of the index sum in each row in calculation mode. The remaining 4T portion constitutes a matching unit. This matching unit uses one bit of the index sum restored from the 6T storage unit as the target number and the corresponding bit of the successively decreasing maximum exponent as the query number, then outputs a query result indicating whether the target number matches the query number.

[0088] It is common knowledge for those skilled in the art that a 6T storage unit can realize data storage and read, write and retain data, and this embodiment will not be described in detail. The following only explains the operation logic of the matching unit in this embodiment to realize data matching.

[0089] This embodiment uses a single 10T-SRAM cell to match a single-bit query number with a single-bit target number, or to match a multi-bit query number with one bit of a multi-bit target number, within a single cycle. Using a row of 10T-SRAM cells, a complete match between a multi-bit query number and a multi-bit target number can be achieved within a single cycle.

[0090] Specifically, the process of matching the single-bit query number and the single-bit target number in the 10T-SRAM cell is as follows:

[0091] (1) Write back each bit of the sum of the weight and the exponent of the operand to the 6T storage unit in each 10T-SRAM unit to form the target number:

[0092] When Q is high and QB is low, it indicates that the target number is "1"; when Q is low and QB is high, it indicates that the target number is "0".

[0093] (2) Precharge the matching bit lines TL and ML to a high level; then input the encoded query number to the matching unit through the local word lines WLL and WLR:

[0094] In this embodiment, the encoding rule of the query number is: when WLL is high and WLR is low, it indicates that the query number is "1"; when WLL is low and WLR is high, it indicates that the query number is "0".

[0095] (3) Ground the matching bit line TL and determine the matching result based on the level change of the matching bit line ML:

[0096] Specifically, when the query number is "1" and the target number is "1," Q is high, QB is low, WLL is high, and WLR is low. At this point, NM4 is off and NM5 is on; NM6 is off and NM7 is on. In this state, a discharge path is formed between TL and ML through NM5 and NM7, and ML eventually drops to a low level, indicating that the query number and target number match successfully.

[0097] When the query number is "1" and the target number is "0," Q is high, QB is low, WLL is low, and WLR is high. At this point, NM4 is off and NM5 is on; NM6 is on and NM7 is off. In this state, no discharge path can be formed between TL and ML, and ML remains high, indicating that the query number and target number have failed to match.

[0098] When the query number is "0" and the target number is "1", Q is low, QB is high, WLL is high, and WLR is low. At this time, NM4 is turned on and NM5 is turned off; NM6 is turned off and NM7 is turned on. In this state, a discharge path cannot be formed between TL and ML, and ML remains high, indicating that the query number and target number fail to match.

[0099] When the query number is "0" and the target number is "0", Q is low, QB is high, WLL is low, and WLR is high. At this time, NM4 is on and NM5 is off; NM6 is on and NM7 is off. In this state, a discharge path is formed between TL and ML through NM4 and NM6, and ML eventually drops to a low level, indicating that the query number and target number match successfully.

[0100] The truth table corresponding to the above process is shown in the following table:

[0101] Table 1: Truth table of data match operation in 10T-SRAM

[0102]

[0103] In summary, in the 10T-SRAM matching unit designed in this embodiment, if the query number and target number are the same, the ML and TL will be directly connected, forming a discharge path. If the query number and target number are the same, the ML and TL will not be connected and no discharge path will be formed. The existence of a connected path will ultimately affect the level of the ML. The change in the ML level can be used to determine whether the query number and target number match successfully.

[0104] The index and array provided in this embodiment are arranged by multiple 10T-SRAM arrays, such as Figure 4 As shown, in this array, for each 10T-SRAM cell in the same row, the matching bit line TL of the leftmost 10T-SRAM cell serves as an independent port, and the matching bit line ML of the rightmost 10T-SRAM cell serves as an independent port, and the ML end of each 10T-SRAM cell is connected to the TL end of the adjacent 10T-SRAM cell on the right.

[0105] In this circuit design, if a multi-bit query number and a multi-bit target number are matched, the target number is simply input bit by bit into the 6T storage cell portion of each 10T-SRAM cell, and then the target number is input bit by bit into the matching cell within each 10T-SRAM cell. If the query number and target number are exactly the same, the TL and ML bits are fully connected, forming a discharge path, and the ML level drops, indicating a successful match between the two multi-bit numbers. However, if any bit in the query number and target number differs, the match fails and the 10T-SRAM cell is disconnected. This results in a disconnection between the TL and ML bits, preventing the discharge path from forming. The ML level remains high, indicating a failed match between the two multi-bit numbers.

[0106] In the solution of this embodiment, Figure 5 As shown, the weight mantissa array is composed of multiple 8T-SRAM units; each 8T-SRAM unit includes two PMOS transistors P1 to P2 and six NMOS transistors N1 to N6. The circuit connection relationship is:

[0107] P1, P2, N1, and N2 are anti-phase cross-coupled to form a set of storage nodes Q and QB. N3 acts as a transmission transistor between Q and bit line BL, and N4 acts as a transmission transistor between QB and bit line BLB. The gates of N3 and N4 are connected to word line W. The gate of N6 is connected to QB, the source of N6 is grounded, and the drain of N6 is connected to the drain of N5. The gate of N5 is connected to the computing word line RWL, and the source of N5 is connected to the computing bit line CL.

[0108] The weight mantissa array uses 8T-SRAM cells, with P1-P2 and N1-N4 forming a 6T storage unit. This 6T storage unit is used for data storage in storage mode and for pre-storing one bit of the mantissa in the weight in the calculation module. N5 and N6 form the multiplication unit.

[0109] In the solution of this embodiment, each 8T-SRAM unit in the weight mantissa array needs to store the mantissa of the weight in addition to pre-storing the mantissa of the weight. It also needs to calculate the product of the mantissa of the operand and the weight according to the mantissa of the input operand. In this embodiment, the storage unit cooperates with N5 and N6 to realize the multiplication operation. The operation logic of this process is as follows:

[0110] (1) The tail part of the weight is pre-stored in the storage unit:

[0111] When Q is high and QB is low, the corresponding bit of the mantissa of the weight is "1"; when Q is low and QB is high, the corresponding bit of the mantissa of the weight is "0".

[0112] (2) Precharge the calculation bit line CL to a high level, and then input the mantissa of the operand through the calculation word line RWL:

[0113] When RWL is high, it indicates that the corresponding bit of the mantissa of the input operand is "0". When RWL is low, it indicates that the corresponding bit of the mantissa of the input operand is "1".

[0114] (3) Output the final calculation result of the mantissa product according to the level change of the calculated bit line CL:

[0115] In this embodiment, when CL drops to a low level, it indicates that the multiplication result is "1", and when CL remains in a high level state, it indicates that the multiplication result is "0".

[0116] Specifically, when RWL is low, Q is high, and QB is low, both N5 and N6 are turned on. At this time, the bit line CL forms a discharge path and drops to a low level, completing the operation "1×1=1".

[0117] When RWL is low, Q is low, and QB is high, N5 is turned on and N6 is turned off. At this time, the bit line CL cannot form a discharge path and remains high, completing the operation "0×1=0".

[0118] When RWL is high, Q is high, and QB is low, N5 is off and N6 is on. At this time, the bit line CL cannot form a discharge path and remains high, completing the operation "1×0=0".

[0119] When RWL is high, Q is low, and QB is high, N5 and N6 are both off. At this time, the bit line CL cannot form a discharge path and remains high, completing the operation "0×0=0".

[0120] The truth table corresponding to the above process is shown in the following table:

[0121] Table 2: Truth table of multiplication operation in 8T-SRAM

[0122]

[0123] In the solution of this embodiment, in addition to data storage, the exponent array also needs to have a data matching function; in addition to data storage, the weight mantissa array also needs to have a multiplication function. The weight exponent only needs to have the function of pre-storing the exponential part of the weight during the calculation process. Therefore, the weight exponent array can use any SRAM unit with data storage function. Of course, the SRAM unit in the weight exponent array can also use the aforementioned 8T-SRAM and 10T-SRAM.

[0124] In the SRAM-based floating-point multiplication and accumulation fast calculation circuit provided in this embodiment, to further shorten the time required to complete floating-point multiplication and accumulation operations, this embodiment borrows the pipeline strategy used in industrial production to design the circuit and perform the calculation tasks during circuit operation. The principle of the pipeline strategy in this embodiment is to divide the entire circuit into an exponential calculation part including an exponent input unit, an exponent sum array, a weighted exponent array, and an adder array; and a mantissa calculation part including a mantissa input module, a weighted mantissa array, a shift register, an adder, a subtraction counter, a normalization module, and a maximum value search module.

[0125] During the multiplication-accumulation process of floating-point numbers, the exponential calculation component completes the exponential summing process for each round of computational tasks in the previous cycle. The mantissa calculation component, in a later cycle, uses the already calculated exponential sums to complete the exponent alignment, mantissa offset, and synthesis of the final multiplication-accumulation result. Because the circuit modules within the exponential and mantissa calculation components of the circuit designed in this embodiment can operate independently and synchronously, the circuit designed in this embodiment can simultaneously complete all operations other than the exponential summing process in the previous round while executing the exponential summing operation in the current round, thereby efficiently utilizing each clock cycle.

[0126] Based on this unique pipeline strategy, the floating-point fast multiplication-accumulation circuit of this embodiment distributes a complete multiplication-accumulation task into two cycles. Furthermore, since two "steps" in different rounds of operations can be completed simultaneously in each cycle, after multiple multiplication-accumulation tasks are completed, the average time required for each multiplication-accumulation operation is only one cycle.

[0127] Specifically, such as Figure 6 As shown, the process of implementing the multiplication and accumulation operation by the SRAM-based floating-point multiplication and accumulation fast operation circuit provided in this embodiment is as follows:

[0128] S1: Initial stage:

[0129] In the first cycle, the exponential input module, the weight exponential array and the adder array are used to complete the input and addition of the exponential parts of each operand and weight in the first round of operation tasks to obtain the exponential sum.

[0130] The maximum value search module reads the exponents and calculation results of each row in the adder array bit by bit in order from low to high.

[0131] S2: Operation stage:

[0132] In the second cycle, the mantissa input module and the weight mantissa array are used to complete the input and multiplication of the mantissa parts of each operand and weight in the first round of calculation tasks to obtain each mantissa product.

[0133] The maximum value finding module writes the calculated result of the read exponent sum back to the exponent sum array and determines the maximum exponent at the same time. The maximum exponent is then decremented by a subtraction counter and compared with each exponent sum in the exponent sum array. After each comparison, the shift register shifts the mantissa product of the exponent sum that is smaller than the maximum exponent.

[0134] The adder tree then adds the mantissa products of the shifted rows to obtain the mantissa sum. Finally, the normalization module generates the corresponding multiply-accumulate result based on the determined maximum exponent and mantissa sum.

[0135] At the same time, in the second cycle, the adder array is also used to complete the input and addition of the exponential parts of each operand and weight in the second round of calculation tasks, and the maximum value search module reads the calculation results of the exponential sum of each row in the adder array in order from low to high.

[0136] S3: Cycle phase:

[0137] Following the same logic, in each subsequent cycle, the multiplication and accumulation results of the previous round are calculated and output, and the addition and caching of the exponential part of the shift round operation are completed;

[0138] Similarly, in the Nth cycle, the processing and output of the N-1th round of multiplication and accumulation operations can be achieved.

[0139] The primary function of the maximum value finding module in the floating-point multiply-accumulate fast calculation circuit provided in this embodiment is to compare the sums of the exponents obtained in each row during each round of multiply-accumulate calculations and determine the maximum exponent among these sums. The maximum exponent determined by the maximum value finding module is later used to align the sums of the exponents in each row and shift the mantissa products.

[0140] To achieve the required functions, the maximum value search module provided in this embodiment includes a plurality of ping-pong units and a plurality of comparison units corresponding to the number of rows in the SRAM array; each ping-pong unit includes register 1 and register 2. The functions of the ping-pong unit and the comparison unit are as follows:

[0141] In the previous cycle, the ping-pong unit is used to read the exponential sums calculated by the adder array in order from low to high and store them in register one; and write each bit of data in register one to register two bit by bit, and the data written into register two is restored to the exponential sum arranged from high to low.

[0142] In the next cycle, the ping-pong unit writes the data in register 2 back to the corresponding row in the index and array, and simultaneously determines the maximum exponent during the write-back process. At the same time, the ping-pong unit also performs the index and read in the next round of calculation and updates register 1.

[0143] This embodiment utilizes a ping-pong unit that simultaneously writes the exponent and calculation result of the subsequent cycle bit by bit in ascending order, and reads the exponent and calculation result of the previous cycle bit by bit in descending order. Therefore, the ping-pong unit serves as a link between the two major "processes" of the floating-point multiplication and accumulation task in this embodiment, allowing the entire calculation process to run continuously and smoothly.

[0144] The function of the comparison unit in the maximum value search module is to compare the exponents of each row after writing back with the operation results to determine the maximum exponent. Figure 7 As shown, in the process of writing back the sum of the exponents of each row, the maximum value finding module of this embodiment determines the maximum exponent by comparing the exponents bit by bit. The process is as follows:

[0145] (1) The exponential sum calculated by the adder array is written bit by bit from register 2 to the exponential sum array in descending order.

[0146] (2) When writing each bit of the exponent sum, determine the current value and make the following decisions:

[0147] A. If the values ​​of the current positions of the sum of the exponents in different rows include both 1 and 0, the numbers whose current positions are 1 are retained and the numbers whose current positions are 0 are removed.

[0148] B. If the values ​​of the current bits of the sums of the exponents in different rows are both 1 or 0, then the numbers on all rows are retained.

[0149] (3) According to the strategy in the previous step, if there is only one number left or the writing of the lowest bit of the exponent sum has been completed, the number finally retained is the maximum exponent.

[0150] For example, assume that the two exponential sums calculated by the adder array are 4-bit binary numbers A and B, whose values ​​are 1010 and 1011 respectively. The bit-by-bit comparison process is as follows:

[0151] Compare the highest bit: A[3]=1, B[3]=1. If these bits are the same, keep both numbers and continue to compare the next bit.

[0152] Compare the second highest bit: A[2]=0, B[2]=0. If these bits are the same, keep both numbers and continue to compare the next bit.

[0153] Compare the third highest bit: A[1]=1, B[1]=0. If these bits are the same, keep both numbers and continue to compare the next bit.

[0154] Compare the fourth highest bit: A[0]=0, A[0]=1, this bit is different, since B[0]>A[0], so B>A, retain B. At this point, the comparison has reached the lowest bit, and the remaining B is the maximum value.

[0155] As another example, suppose the adder array outputs four 4-bit binary exponent sums, each with the following values:

[0156] A=1010 (decimal 10)

[0157] B=1001(decimal 9)

[0158] C=1100 (decimal 12)

[0159] D=0111(decimal 7)

[0160] First, start comparing from the highest bit. If the highest bit of the number is 0, exclude the number. If the highest bit is 1, keep the number. After this step, retain: A, B, C.

[0161] Continue to compare the second highest bit: if the second highest bit is 0, exclude it, and if it is 1, retain it, so this step only leaves C. At this point, although the comparison has not reached the lowest bit, there is only one number left, so the retained number C is the maximum value.

[0162] In the circuit of this embodiment, the exponent sum array, the subtraction counter and the shift register shift the mantissa product according to the maximum exponent output by the maximum value search module. Figure 8 and Figure 9 , the operation strategy for implementing exponent sum alignment and mantissa product shift in this embodiment is as follows:

[0163] The subtraction counter decrements the maximum exponent by 1 in each cycle and sends it to the exponent sum array.

[0164] The index and array matches the maximum index at the current moment with the index and index stored in itself in each cycle. If the match is successful, a corresponding pairing flag is generated and sent to the shift register.

[0165] The shift register is used to shift the exponent and the corresponding mantissa product of each row in each cycle, and stop the shift operation of the mantissa product of the corresponding row when any pairing flag is received in each cycle.

[0166] When the preset maximum number of cycles is reached, the shifting of all mantissa products is completed.

[0167] Example 2

[0168] Based on the embodiment scheme, this embodiment further provides a CIM chip that integrates the SRAM-based floating-point multiplication-accumulation fast calculation circuit of Example 1. The CIM chip has a storage mode and a calculation mode. In the storage mode, the CIM chip functions as a memory. In the calculation mode, the CIM chip is used to implement multiplication-accumulation operations between multiple groups of multi-bit floating-point operands and multi-bit floating-point weights.

[0169] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A floating-point multiplication and accumulation fast operation circuit based on SRAM, characterized in that: The invention is based on an SRAM array and its peripheral circuit design, and is used to implement multiplication and accumulation operations between multiple groups of multi-bit floating-point operands and multi-bit floating-point weights. The SRAM array in the floating-point multiplication and accumulation fast operation circuit is divided into three parts by column, namely, an exponent sum array, a weight exponent array, and a weight mantissa array. The floating-point multiplication-accumulation fast operation circuit further comprises: an exponent input module, a mantissa input module, an adder array, a maximum value search module, a subtraction counter, a shift register, an adder tree and a normalization module; Each row in the weight exponent array is used to pre-store the exponent part of the weight in bitwise manner; the exponent input module is used to input the exponent part of the operand into each row in the adder array; the adder array is used to read the stored value of the weight exponent array and calculate the exponential sum of the weight and the operand; the mantissa input module is used to input the mantissa part of the operand into each row in the weight mantissa array; each row in the weight mantissa array is used to pre-store the mantissa part of the weight in bitwise manner, and uses its own logical operation function to perform a multiplication operation on the mantissa part of the weight and the operand to obtain a mantissa product; The maximum value search module is used to first obtain the calculation results of the exponential sum of each row output by the adder array, and determine the maximum exponent while writing the exponential sum back to the exponential sum array bit by bit; then, in combination with the subtraction counter and the exponential sum array, determine the bit difference between the exponent sum calculated in each row and the maximum exponent, and shift the mantissa product calculated in the corresponding row according to the bit difference by the shift register; the maximum value search module includes a plurality of ping-pong units and a plurality of comparison units corresponding to the number of rows of the SRAM array; each ping-pong unit includes register 1 and register 2; in the previous cycle, the ping-pong unit The ping-pong unit is configured to read the exponential sums calculated by the adder array in order from low to high and store them in register one; and write each bit of data in register one to register two, so that the data written into register two is restored to the exponential sum arranged from high to low; in the next cycle, the ping-pong unit writes back the data in register two to the corresponding row in the exponential sum array, and synchronously determines the maximum exponent during the write-back process; the ping-pong unit also performs the exponential sum reading in the next round of calculation and updates register one; the comparison unit is configured to compare the exponential sum calculation results of each row after the write-back to determine the maximum exponent; The adder tree is used to add the mantissa products of each row after the shift is completed to obtain the mantissa sum; finally, the normalization module generates a corresponding multiplication-accumulation operation result according to the maximum exponent and the mantissa sum.

2. The SRAM-based floating-point multiplication-accumulation fast operation circuit according to claim 1, wherein: It uses a pipeline strategy to complete the multiplication and accumulation tasks of floating-point numbers. The process is as follows: S1: Initial stage: In the first cycle, the exponential input module, weight exponent array and adder array are used to complete the input and addition of the exponential parts of each operand and weight in the first round of operation tasks to obtain the exponential sum; The maximum value search module reads the exponent and calculation result of each row in the adder array bit by bit in order from low to high; S2: Operation stage: In the second cycle, the mantissa input module and the weight mantissa array are used to complete the input and multiplication of the mantissa parts of each operand and weight in the first round of calculation tasks to obtain each mantissa product; The maximum value finding module writes back the calculated result of the read exponential sum to the exponent sum array and determines the maximum exponent while writing back; then, the maximum exponent is decremented by a subtraction counter and compared with each exponential sum in the exponent sum array; and after each comparison, the shift register is used to shift the mantissa product of the exponent sum smaller than the maximum exponent; Then, the adder tree adds the mantissa products of each shifted row to obtain a mantissa sum; finally, the normalization module generates a corresponding multiplication-accumulation operation result according to the maximum exponent and the mantissa sum; At the same time, in the second cycle, the adder array is also used to complete the input and addition of the exponential parts of each operand and weight in the second round of calculation tasks, and the maximum value search module reads the calculation results of the exponential sum of each row in the adder array in order from low to high; S3: Cycle phase: Following the same logic, in each subsequent cycle, the multiplication and accumulation results of the previous round are calculated and output, and the addition and caching of the exponential part of the shift round operation are completed; Similarly, in the Nth cycle, the processing and output of the N-1th round of multiplication and accumulation operations can be achieved.

3. The SRAM-based floating-point multiplication-accumulation fast operation circuit according to claim 1, wherein: The method by which the maximum value finding module determines the maximum exponent in the process of writing back the sum of the exponents of each row is: (1) The exponential sum calculated by the adder array is written bit by bit from register 2 to the exponential sum array in descending order; (2) When writing each bit of the exponent sum, determine the current value and make the following decisions: A. If the exponents and the current values ​​of different rows contain both 1 and 0, the numbers with the current digit being 1 are retained and the numbers with the current digit being 0 are discarded. B. If the current positions of the sum of the exponents in different rows are all 1 or all 0, keep the numbers on all rows; (3) According to the strategy in the previous step, if there is only one number left or the writing of the lowest bit of the exponent sum has been completed, the number finally retained is the maximum exponent.

4. The SRAM-based floating-point multiplication-accumulation fast operation circuit according to claim 1, wherein: The operation strategy of the exponent sum array, subtraction counter and shift register to shift the mantissa product according to the maximum exponent output by the maximum value finding module is as follows: The subtraction counter performs a subtraction operation on the maximum exponent by 1 in each cycle and then sends the subtraction operation to the exponent sum array; The index sum array matches the maximum index at the current moment with the index sum stored in itself in each cycle, and if the match is successful, a corresponding pairing flag is generated and sent to the shift register; The shift register is used to shift the exponent and the corresponding mantissa product of each row in each cycle, and stop the shift operation of the mantissa product of the corresponding row when any pairing flag is received in each cycle; When the preset maximum number of cycles is reached, the shifting of all mantissa products is completed.

5. The SRAM-based floating-point multiplication-accumulation fast operation circuit according to claim 1, wherein: The weight mantissa array is composed of multiple 8T-SRAM units; each 8T-SRAM unit includes two PMOS transistors P1 to P2 and six NMOS transistors N1 to N6; the circuit connection relationship is: P1, P2, N1, and N2 are anti-phase cross-coupled to form a group of storage nodes Q and QB; N3 serves as a transmission tube between Q and the bit line BL, and N4 serves as a transmission tube between QB and the bit line BLB. The gates of N3 and N4 are connected to the word line WL; the gate of N6 is connected to QB, the source of N6 is grounded, and the drain of N6 is connected to the drain of N5; the gate of N5 is connected to the computing word line RWL; and the source of N5 is connected to the computing bit line CL.

6. The SRAM-based floating-point multiplication-accumulation fast operation circuit according to claim 5, wherein: The P1-P2 and N1-N4 parts of the 8T-SRAM unit constitute a 6T storage unit. The 6T storage unit in the 8T-SRAM unit is used to implement the data storage function in the storage mode and to pre-store one of the mantissa parts of the weight in the calculation module; The operation logic of the storage unit in conjunction with N5 and N6 to implement multiplication is as follows: (1) The tail part of the weight is pre-stored in the storage unit: When Q is high and QB is low, the corresponding bit of the mantissa of the weight is "1"; when Q is low and QB is high, the corresponding bit of the mantissa of the weight is "0"; (2) Precharge the calculation bit line CL to a high level, and then input the mantissa of the operand through the calculation word line RWL: When RWL is high, it means that the corresponding bit of the mantissa of the input operand is "0", and when RWL is low, it means that the corresponding bit of the mantissa of the input operand is "1"; (3) Output the final calculation result of the mantissa product according to the level change of the calculated bit line CL: When CL drops to a low level, it indicates that the product result is "1"; when CL remains high, it indicates that the product result is "0".

7. The SRAM-based floating-point multiplication-accumulation fast operation circuit according to claim 1, wherein: The index and array are composed of multiple 10T-SRAM cells; each 10T-SRAM cell includes two PMOS transistors PM0 to PM1 and eight NMOS transistors NM0 to NM7, and the circuit connection relationship is: The sources of PM0 and PM1 are connected to VDD; the drain of PM0 is connected to the gates of PM1, NM3, NM4, and the drains of NM0 and NM2, and serves as the storage node Q; the drain of PM1 is connected to the gates of PM0, NM2, NM5, and the drains of NM1 and NM3, and serves as the inverting storage node QB; the sources of NM2 and NM3 are grounded; the source of NM0 is connected to the bit line BL; the source of NM1 is connected to the bit line BLB; the gates of NM0 and NM1 are connected to the word line WL; The drain of NM4 and the source of NM5 are connected to the matching bit line TL; the source of NM4 and the drain of NM6 are connected; the drain of NM5 and the source of NM7 are connected; the source of NM6 and the drain of NM7 are connected to the matching bit line ML; the gate of NM6 is connected to the local word line WLL; the gate of NM7 is connected to the local word line WLR.

8. The SRAM-based floating-point multiplication-accumulation fast operation circuit according to claim 7, wherein: The PM0-PM1 and NM0-NM3 parts of the 10T-SRAM unit constitute a 6T storage unit; the 6T storage unit in the 10T-SRAM unit is used to implement data storage in storage mode and to restore one bit of the exponential sum in each row in calculation mode; the remaining 4T part constitutes a matching unit; the matching unit uses one bit of the exponential sum restored in the 6T storage unit as the target number; the corresponding bit of the successively decreasing maximum exponent is used as the query number, and then outputs a query result to determine whether the target number matches the query number; The operation logic of the matching unit to achieve data matching is as follows: (1) Write back each bit of the sum of the weight and the exponent of the operand to the 6T storage unit in each 10T-SRAM unit to form the target number: When Q is high and QB is low, it means the target number is "1"; when Q is low and QB is high, it means the target number is "0"; (2) Precharge the matching bit lines TL and ML to a high level; then input the encoded query number to the matching unit through the local word lines WLL and WLR: When WLL is high and WLR is low, it means the query number is "1"; when WLL is low and WLR is high, it means the query number is "0"; (3) Ground the matching bit line TL and determine the matching result based on the level change of the matching bit line ML: When the query number and target number are the same, a discharge path is formed between ML and TL, and ML drops to a low level, indicating that the two are matched successfully; When the query number and the target number are different, a discharge path cannot be formed between ML and TL, and ML remains at a high level, indicating that the matching between the two has failed.

9. A CIM chip, characterized in that: It integrates the SRAM-based floating-point multiplication and accumulation fast operation circuit according to any one of claims 1 to 8; the CIM chip has a storage mode and a calculation mode; in the storage mode, the CIM chip is used as a memory; in the calculation mode, the CIM chip is used to implement multiplication and accumulation operations between multiple groups of multi-bit floating-point operands and multi-bit floating-point weights.

Images