Index and normalization circuit, maximum value search circuit, MAC circuit and chip

By designing the combination of addition array, data transmission module and maximum value search circuit, efficient processing of floating point index and normalization is achieved, solving the problems of high area overhead and power consumption in floating point MAC operations in traditional circuits, and improving Computational efficiency and flexibility.

CN119045778AActive Publication Date: 2024-11-29ANHUI UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411143115.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-29
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

Traditional floating-point MAC operation circuits have problems of high area overhead, high power consumption and insufficient efficiency when handling exponential and normalization tasks.

Method used

An exponential and normalization circuit including an addition array, a data transmission module and a maximum value search circuit is designed. The floating point exponential is added through the addition array. The data transmission module realizes the reverse order transmission of data, and the maximum value search circuit recognizes it. The maximum value is and the normalized result is output through the output module to achieve pipelined operation.

Benefits of technology

It reduces the delay of the computing process, improves the computing rate, reduces area overhead and energy loss, simplifies timing control, and improves computing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045778A_ABST
    Figure CN119045778A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of integrated circuits, and particularly relates to an index and normalization circuit, a maximum value search circuit, an MAC circuit, a chip and a CIM chip integrated with a floating point type MAC circuit. Wherein the maximum value searching circuit is composed of a plurality of comparison units which are arranged in columns, and each comparison unit comprises three NMOS (N-channel Metal Oxide Semiconductor) transistors N1-N3, a PMOS (P-channel Metal Oxide Semiconductor) transistor P1, an AND gate AND1, an OR gate OR1 and a phase inverter INV1. The circuit adopts a cross structure design, the circuit is simpler, and the recognition speed is higher. And the index and normalization circuit comprises an addition array, a data transmission module, a maximum value search circuit and an output module, and can sequentially complete a plurality of processes in an operation process by adopting a streamline strategy, and perform parallel processing on part of the processes, so that the delay in the whole task is shortened, an index normalization task is processed more efficiently, and the operation efficiency is improved. And the area overhead and the power consumption level of the circuit are reduced. The problem that in the prior art, an index and normalization special circuit lacks is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of integrated circuits, and in particular relates to an exponential and normalization circuit, a maximum value search circuit, a floating-point MAC circuit, and a CIM chip integrated with the floating-point MAC circuit. Background Art

[0002] In recent years, the rapid development of deep learning and neural networks has put forward higher requirements for computing power. The traditional von Neumann architecture, which uses a processor and memory separation model, faces serious challenges in processing large-scale neural networks. As the scale of neural network models continues to expand, the frequent movement of data between the processor and memory has become a major bottleneck for performance and energy consumption. This problem is particularly prominent in deep learning tasks that require processing massive amounts of data. In order to overcome these limitations, in-memory computing (IMC) technology has attracted widespread attention in the field of deep learning. In-memory computing allows computing operations to be performed directly in the storage unit without frequently transferring data between the processor and memory. This architecture is of great significance for neural network computing.

[0003] It is worth noting that in the neural network calculation process, floating point exponent and normalization (Asynchronous Exponent Normalization 15Processing, AENP) is an important process of neural network floating point calculation. Floating point exponent and normalization play a key role in ensuring the calculation accuracy of neural network, improving calculation efficiency, enhancing model generalization ability and supporting various optimization technologies.

[0004] In traditional logic circuits with floating-point MAC operation functions, the high-parallel data reading, addition, and maximum value search processes required for the exponentiation and normalization of floating-point numbers are completed sequentially in multiple cycles through different circuit modules. This results in poor performance of such logic circuits in terms of area overhead and power consumption. Therefore, how to provide an exponentiation and normalization circuit with higher integration and operation efficiency, lower power consumption, and faster speed is becoming a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] In order to solve the problems of area overhead, power consumption and insufficient efficiency when the existing circuit processes the exponentiation and normalization tasks in the floating-point MAC operation, the present invention provides an exponentiation and normalization circuit, a maximum value search circuit, a floating-point MAC circuit, and a CIM chip integrated with the floating-point MAC circuit.

[0006] The technical solution provided by the present invention is as follows:

[0007] The invention discloses an exponent sum normalization circuit, which is used for adding the exponent parts of multiple groups of floating-point operands and floating-point weights in floating-point multiplication and accumulation operations, and outputting the maximum value of each exponent sum calculated.

[0008] The index and normalization circuit includes: an addition array, a data transmission module, a maximum value search circuit and an output module. The addition array includes a plurality of addition units arranged in columns; each row of the addition units is used to convert the input operand index IE i Weight index WE i Add and output the corresponding exponent and SUM i .

[0009] The data transmission module is composed of a plurality of stack units arranged in columns; each stack unit is used to receive the exponent and SUM output by the addition unit bit by bit in the order from low to high. i After receiving the data on each bit, it is output in reverse order to the maximum value search circuit.

[0010] The maximum value search circuit is used to input multiple exponentials and SUM i Generates a signal LOG representing the maximum index contained therein i The maximum value search circuit is composed of multiple comparison units arranged in columns, each comparison unit includes three NMOS tubes N1 to N3, one PMOS tube P1, one AND gate AND1, one OR gate OR1, and one inverter INV1. The circuit connection relationship of the maximum value search circuit is as follows:

[0011] In each comparison unit, the source of N1 is grounded; the drain of N1 is connected to the source of N2; the drain of N2 is connected to the source of N3; the drains of N3 and P1 are connected to one of the input terminals of AND1 and serve as the output port of the mark signal of the corresponding row. The source of P1 is connected to VDD; the gate of N1 is connected to the enable signal EN; the gate of P1 is connected to the PRE port; the other input terminal of AND1 and the input terminal of INV1 are connected to the output terminal of the stack unit of the corresponding row; the output terminal of AND1 is connected to one of the output terminals of OR1. In the entire maximum value search circuit, the other input terminal of OR1 in the comparison unit of the first row is grounded, and the other input terminal of OR1 in the comparison units of the remaining rows is connected to the output terminal of OR1 in the comparison unit of the previous row; the output terminal of OR1 in the comparison unit of the last row is connected to the gate of N3 in the comparison units of all rows.

[0012] The output module is used to receive the exponents and SUM output by the addition unit. i The flag signal LOG output by the comparison unit i, and the index and output corresponding to the flag signal that is still high after multiple rounds of comparison are used as the normalized result SUM.

[0013] As a further improvement of the present invention, each adding unit in the adding array is composed of a serial adder FA and a D flip-flop; the two data input terminals of FA are respectively connected to IE i WE i , the output end of FA is connected to the input end of the stack unit; the carry input end CI of FA is connected to the data output end Q of the D flip-flop, the carry output end CO of FA is connected to the data input port D of the D flip-flop, and the clock signal port of the D flip-flop is connected to the clock signal CLK.

[0014] As a further improvement of the present invention, the maximum number of bits supported by the stack unit in the data transmission module is determined by the exponent output by the addition unit and the sum i When the exponent output by the addition unit is equal to SUM i When it is N bits, the data transmission module uses an N-bit stack unit.

[0015] As a further improvement of the present invention, the N-bit stack unit is composed of N D flip-flops D1 to Dn, 2n-2 tri-state buffers E1 to E2n-2 and a 2-to-1 selector MUX, and the circuit connection relationship is as follows:

[0016] D1~Dn are forward cascaded through E1~En-1, the Q end of the previous stage D flip-flop is connected to the input end of the three-state buffer, and the output end of the three-state buffer is connected to the D end of the next stage D flip-flop; the control end of E1~En-1 is connected to the first enable signal EN1.

[0017] D1~Dn are also reverse cascaded through En~En-2, the D end of the next stage D flip-flop is connected to the input end of the three-state buffer, and the output end of the three-state buffer is connected to the Q end of the previous stage D flip-flop; the control end of En~En-2 is connected to the second enable signal EN2.

[0018] The clock signal ports of D1 to Dn-2 are connected to the clock signal CLK; the D end of the first-stage D flip-flop is connected to the output end of the addition unit, and is used to receive the exponents and SUM output by the addition unit in reverse order. i .

[0019] The data port 0 of the MUX is grounded, the data port 1 is connected to the Q terminal of the first-stage D flip-flop, the selection terminal is connected to the OPT signal, and the output terminal outputs the exponential and SUM arranged in sequence. i .

[0020] As a further improvement of the present invention, the stack unit implements the bit-by-bit inversion and output of the received N-bit data, which includes two stages: data storage and data reading. The process is as follows:

[0021] (1) Data storage stage

[0022] Set EN1 to high level and EN2 to low level; and control the MUX to output the data of data port 0 through the OPT signal.

[0023] N bits of data are input from the low bit to the high bit through the D terminal of D1. After several clock cycles, the low bit to the high bit of the N bits of data are stored in Dn to D1 in sequence. At this stage, the output of MUX is 0.

[0024] (2) Data readout stage

[0025] Set EN1 to high level and EN2 to low level; and control the MUX to output the data of data port 1 through the OPT signal.

[0026] In the subsequent clock cycles, the data stored in D1 to Dn are outputted in sequence through the output port of MUX; that is, MUX outputs N-bit data arranged from high to low, realizing bit-by-bit inversion of the data input to the D end of D1 in the data storage stage.

[0027] As a further improvement of the present invention, the maximum value search circuit realizes the strategy of identifying the maximum value from the exponential sum of each N bit input and generating the corresponding flag signal as follows:

[0028] (1) The respective flag signal lines LOG are connected through the PRE ports in the comparison units of each row. i Precharged to high level.

[0029] (2) Receive the indexes and SUMs output by the stack cells in each row in order from high to low. i .

[0030] (3) The maximum value search circuit receives the index and SUM calculated for each row i When each bit is set, the level state of the mark signal line of each row changes and satisfies the following rules:

[0031] A. If the SUM entered in each row i If the corresponding bit of includes both "1" and "0", the flag signal line of the corresponding row with data "1" maintains the previous level state, and the flag signal line of the corresponding row with data "0" discharges and becomes a low level.

[0032] B. When the flag signal line of any row is at a low level, it will always remain at a low level and will no longer become a high level.

[0033] C. If the SUM entered in each row iIf the corresponding bits are all "1" or all "0", the mark signal lines of each row maintain the previous level state.

[0034] (4) After any cycle, if only one row of the flag signal line is at a high level, the exponential sum of the input of this row is the maximum value of all exponential sums, which is taken as the normalization result;

[0035] or,

[0036] After completing the input of the lowest bit of the index sum of each row, the index sum of the corresponding row whose flag signal line is still at a high level is the maximum value among all the index sums, which is used as the normalization result.

[0037] As a further improvement of the present invention, when the addition array, the data transmission module, and the maximum value search circuit include M rows, the output module used by the index and normalization circuit also includes M transmission gates arranged in columns and a cascaded OR gate; the transmission gate TG of each row i The input terminal of SUM i , the control end connects to LOG i The cascaded OR gate uses the non-zero output of each transmission gate as the final normalized result SUM.

[0038] The present invention also includes a maximum value search circuit, which is used to identify the maximum value from a plurality of multi-bit numbers input bit by bit. The maximum value search circuit is composed of a plurality of comparison units arranged in columns, each comparison unit includes 3 NMOS tubes N1 to N3, 1 PMOS tube P1, 1 AND gate AND1, 1 OR gate OR1, and an inverter INV1. In each comparison unit, the source of N1 is grounded; the drain of N1 is connected to the source of N2; the drain of N2 is connected to the source of N3; the drains of N3 and P1 are connected to one of the input terminals of AND1 and serve as the output port of the mark signal of the corresponding row. The source of P1 is connected to VDD; the gate of N1 is connected to the enable signal EN; the gate of P1 is connected to the PRE port; the other input terminal of AND1 is connected to the input terminal of INV1 and serves as the input port of each multi-bit number. The output end of AND1 is connected to one of the output ends of OR1; in the entire maximum value search circuit, the other input end of OR1 in the comparison unit of the first row is grounded, and the other input end of OR1 in the comparison unit of the remaining rows is connected to the output end of OR1 in the comparison unit of the previous row; the output end of OR1 in the comparison unit of the last row is connected to the gate of N3 in the comparison units of all rows.

[0039] The present invention also includes a floating-point MAC circuit, which is used to implement multiplication and accumulation operations between multiple groups of floating-point numbers. The MAC circuit uses the exponent sum normalization circuit as described above to normalize the exponent sum of each group of floating-point numbers, and outputs the maximum value of all calculated exponent sums as the normalization result.

[0040] The present invention also includes a CIM chip having a floating-point MAC operation function, wherein the CIM chip is integrated with the aforementioned floating-point MAC circuit.

[0041] The technical solution provided by the present invention has the following beneficial effects:

[0042] The present invention provides a new maximum value search circuit with a cross structure, which can quickly identify the maximum value after multiple multi-bit numbers are input. Compared with various existing circuits with the same functions, the maximum value search circuit designed by the present invention has a simpler structure, can greatly reduce area overhead and energy loss, and has a higher recognition rate, allowing more flexible operation scheduling, so it has a wider range of application scenarios.

[0043] On the basis of the maximum value search circuit, the present invention also designs a circuit dedicated to realizing exponentiation and normalization in floating-point MAC operations. The circuit can orderly associate the "exponent addition-maximum exponent search-exponent sum normalization" work in the exponentiation and normalization tasks to realize pipeline operation, thereby realizing exponentiation and normalization quickly in a shorter cycle, reducing the delay of the operation process and improving the operation rate. In addition, compared with the existing modules with the same function, the area overhead and energy loss of the circuit of the present invention are also more optimized. The timing control of the operation process is also simpler. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a circuit diagram of a maximum value search circuit provided in Embodiment 1 of the present invention.

[0045] Figure 2 The maximum value search circuit provided in the first embodiment of the present invention realizes a signal flow chart for identifying the maximum value of four five-bit binary numbers.

[0046] Figure 3 This is a circuit architecture diagram of the exponential and normalization circuit provided in Example 2 of the present invention.

[0047] Figure 4 4 is a circuit diagram of a stack unit in an exponential and normalization circuit according to Embodiment 2 of the present invention.

[0048] Figure 5 4 is a circuit diagram of an output module in an exponential and normalization circuit of Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] Example 1

[0051] The present invention also includes a maximum value search circuit, which is used to identify the maximum value from a plurality of multi-bit numbers input bit by bit. The maximum value search circuit is composed of a plurality of comparison units arranged in columns. In practical applications, each multi-bit number to be identified is synchronously input bit by bit in the comparison units of different rows of the circuit in the order from MSB to LSB. When each multi-bit number is input, the circuit can output a flag signal LOG representing the size of the input data of each row. When the final result of the flag signal LOG of a row is in a high level state, it represents that the multi-bit number input into the comparison unit of the row is the maximum value of all data. When the final result of the flag signal LOG of a row is in a low level state, it represents that the multi-bit number input into the comparison unit of the row is not the maximum value of all input data.

[0052] Specifically, Figure 1 As shown, each comparison unit in the maximum value search circuit provided by this embodiment includes three NMOS tubes N1 to N3, one PMOS tube P1, one AND gate AND1, one OR gate OR1, and one inverter INV1. In each comparison unit, the source of N1 is grounded; the drain of N1 is connected to the source of N2; the drain of N2 is connected to the source of N3; the drains of N3 and P1 are connected to one of the input terminals of AND1 and serve as the output port of the flag signal LOG of the corresponding row. The source of P1 is connected to VDD; the gate of N1 is connected to the enable signal EN; the gate of P1 is connected to the PRE port; the other input terminal of AND1 is connected to the input terminal of INV1 and serves as the input port of each multi-bit number. The output terminal of AND1 is connected to one of the output terminals of OR1.

[0053] In the entire maximum value search circuit, the other input terminal of OR1 in the comparison unit of the first row is grounded, and the other input terminals of OR1 in the comparison units of the remaining rows are connected to the output terminal of OR1 in the comparison unit of the previous row; the output terminal of OR1 in the comparison unit of the last row is connected to the gate of N3 in the comparison units of all rows.

[0054] In this embodiment, the number of rows of comparison units in the maximum value search circuit corresponds to the maximum number of groups of input data supported by the circuit. For example, a maximum value search circuit with N rows of comparison units supports maximum value searches for N multi-bit numbers at most. The circuit does not limit the number of digits of the multi-bit number to be identified, and the circuit can support finding the maximum value from multiple multi-bit numbers of any digit.

[0055] The maximum value search circuit provided in this embodiment realizes the strategy of identifying the maximum value from the sum of the exponents of each N bit input and generating the corresponding flag signal as follows:

[0056] (1) The respective flag signal lines LOG are connected through the PRE ports in the comparison units of each row. i Precharged to high level.

[0057] (2) Through the IN gates in the comparison units of each row i The port synchronously receives the multi-bit number to be recognized in order from high to low.

[0058] (3) When the maximum value search circuit receives each bit of the multi-bit number input from each row, the level state of the mark signal line of each row changes according to the following rules:

[0059] A. If the corresponding bits of the multi-bit numbers input in each row include both "1" and "0", the flag signal line of the corresponding row with data "1" maintains the previous level state, and the flag signal line of the corresponding row with data "0" discharges and becomes a low level.

[0060] B. When the flag signal line of any row is at a low level, it will always remain at a low level and will no longer become a high level.

[0061] C. If the corresponding bits of the multi-bit numbers input in each row are all "1" or all "0", the mark signal line of each row maintains the previous level state.

[0062] (4) Assuming that the number of bits of the multi-bit numbers input in each row is M bits, then after at most M cycles, the flag signal output by the comparison unit in each row can reflect the maximum value therein.

[0063] Specifically, after any cycle, if only the flag signal line of one row is at a high level, the exponential sum of the input of the row is the maximum value of all exponential sums. Alternatively, after completing the input of the lowest bit of the multi-bit number of each row, the exponential sum of the corresponding row whose flag signal line is still at a high level is the maximum value of all exponential sums. It should be noted that in each comparison process, the number of comparison units whose flag signal LOG is still at a high level may be one row or multiple rows. When the characterization signals output by multiple comparison units are all 1, it means that the inputs of the corresponding rows are the same and are all the maximum values ​​of all input data.

[0064] The following describes in detail the search process and working principle of the maximum value search circuit provided in this embodiment by taking four 5-bit numbers as an example:

[0065] Assume that the four binary data RAW1 to RAW4 to be compared in the maximum value search circuit are 11001, 10011, 11011, and 11001. Before the data is input to the comparison units of each row, the gate terminal of the PMOS tube P1 in each comparison unit is set to a high level through the PRE signal line, and then the flag signal line LOG of each row is set to i Precharge to high level state in advance. Figure 2 It reflects the changes of various signals during the data input process. Then, each 5-bit binary number is input into the comparison unit of each row synchronously according to the principle of inputting the MSB first, then the second lowest bit, and finally the LSB.

[0066] 1. Cycle 1

[0067] Combination Figure 2 As shown, in the first cycle, the value input to each row is "1111", that is, the input IN of each row is all "1". At this time, the working status of the MOS tubes in each row is roughly the same. Before the highest bit number enters the comparison unit, the LOG signal line of each row has been precharged to "1". Since the input IN on each bit is also "1", the output of the AND gate is "1". The output of the AND gate is connected to the input of the OR gate, so the OR gate in each row also outputs "1". The output of the OR gate in each row is used as the CAN signal, so the CAN signal is also "1". Since the CAN signal line is electrically connected to the gate of the NMOS tube N3 of each level of the comparison unit, the N3 of each level of the comparison unit is in the on state.

[0068] In the comparison phase of each cycle, EN in each row is turned on, so the NMOS tube N1 in each row is also in the on state. The input data IN in each row is connected to the NMOS tube N2 in each row through the inverter. Since the input IN in each row in this cycle is "1", N2 in each row is in the off state. It can be seen that N1 and N3 are turned on in the comparison unit of each row, but N2 is turned off, so the LOG signal line of each row cannot be grounded through N3, N2, and N1 in sequence. That is, LOG is maintained in a high level state.

[0069] It can be seen that in this cycle, all row inputs are "1", so the LOG flag signals of RAW1 to RAW4 are consistent with the previous cycle (i.e., the precharge stage), and are all high. The change of the LOG signal in this cycle meets rule C.

[0070] 2. Cycle2

[0071] In the second cycle, the input of each row is "1011". At this time, the input value of the comparison unit is "1" in each row, which is consistent with the input value of "1" in the first cycle, that is, the LOG signal line remains "1". It is worth noting that the OR gates in each row are cascaded in sequence and connected to the gate of N3 in each row. Therefore, as long as the OR gate of any row outputs 1, N3 in all rows is turned on. Therefore, N3 in all rows is turned on in this cycle.

[0072] In each cycle, N1 is controlled by EN and remains on, while the on-state of N2 in each row is only affected by the value of the input data IN. When IN is "0", the gate of N2 is at a high level after the inverter, and N2 is turned on. On the contrary, when IN is "1", the gate of N2 is at a low level after the inverter, and N2 is turned off.

[0073] Therefore, in this cycle, N2 of the row corresponding to RAW2 with the second highest bit being "0" is turned on, and N2 of other rows is turned off. This eventually causes the LOG signal line of the row corresponding to RAW2 to form a discharge path through N3, N2, and N1, and to be at a low level. The LOG signal lines of other rows remain at a high level.

[0074] It can be seen that in this cycle, the LOG flag signal of RAW2 with input "0" drops to a low level, while the LOG flag signals of RAW1, RAW3, and RAW4 with input "1" remain at the previous level, that is, high level. The change of the LOG signal in this cycle meets Rule A.

[0075] 3. Cycle3

[0076] In the third cycle, the data input to each row is "0000", that is, all "0" input, at this time, the output of each row's AND gate is "0", which in turn causes the output of the OR gate to also be "0", and finally makes the CAN signal line connected to the N3 gate of each row also "0". At this time, N3 of each row is cut off, that is, the LOG signal line of each row and the ground line cannot form a discharge path, and maintain the previous level state.

[0077] It can be seen that in this cycle, all row inputs are "0", so the LOG flag signals of RAW1 to RAW4 all maintain the level state of the previous cycle. The change of the LOG signal in this cycle meets rule C.

[0078] 4. Cycle4

[0079] In the fourth cycle, the data input to each row is "0110". Since the LOG signal line of the row corresponding to RAW3 and the input IN of the current cycle are both "1", the CAN signal is also "1" and N3 of each row is turned on.

[0080] According to the input data of each row, the gate of N2 of the rows corresponding to RAW1 and RAW4 is high, and N2 is turned on. The gate of N2 of the rows corresponding to RAW2 and RAW3 is low, and N2 is turned off. Since N1 of all rows in this cycle is already in the on state, the rows corresponding to RAW1 and RAW4 form a discharge path, and the LOG signal lines of these two rows also become low.

[0081] At this point, after four cycles, the signal lines of the corresponding rows of RAW1, RAW2 and RAW4 are all reduced to a low level. Therefore, the data finally input to RAW3, whose LOG signal line is still kept at a high level, is the maximum value.

[0082] It can be seen that in this cycle, the inputs of RAW1 and RAW4 are "0", and the inputs of RAW2 and RAW3 are "1", so the flag signal lines LOG of RAW1 and RAW4 maintain the level state of the previous cycle, and the flag signal lines LOG of RAW1 and RAW4 drop to a low level. The change of the LOG signal in this cycle meets rules A and B.

[0083] Cycle5

[0084] In the fifth cycle, the input of each row is "1111", that is, all "1" input. At this time, the input signal IN outputs "0" through the inverter, so the gate voltage of N2 in each row is low, and N2 in all rows is turned off, so that a discharge path cannot be formed, and the mark signal line LOG of each row still maintains the previous level state.

[0085] It can be seen that in this cycle, the inputs of RAW1 to RAW4 are all "1", and the flag signal lines LOG of RAW1 to RAW4 all maintain the level state of the previous cycle. The change of the LOG signal in this cycle meets rules A and B.

[0086] In summary, although the maximum value search circuit provided by this embodiment is very simple, it can identify the maximum value of all input data in real time after multiple multi-bit numbers are input, and the data search performance is extremely strong. This new circuit can effectively reduce the area overhead and energy consumption of traditional maximum value search circuits, and greatly improve the maximum value search efficiency, and has a wide range of application scenarios.

[0087] Example 2

[0088] Utilizing the maximum value search circuit designed in Example 1, this embodiment further provides an exponent sum normalization circuit, which is used to add the exponential parts of multiple groups of floating-point operands and floating-point weights in floating-point multiplication and addition operations, and output the maximum value of each exponential sum calculated.

[0089] like Figure 3As shown, the index and normalization circuit includes: an addition array, a data transmission module, a maximum value search circuit and an output module. The addition array includes a plurality of addition units arranged in columns; each row of the addition units is used to convert the input operand index IE i Weight index WE i Add and output the corresponding exponent and SUM i .

[0090] Specifically, each adding unit in the adding array in this embodiment uses an adder with carry, and is composed of a serial adder FA and a D flip-flop. In each row of the adding unit, the two data input terminals of FA are connected to IE i WE i , the output of FA is connected to the input of the stack unit; the carry input CI of FA is connected to the data output Q of the D flip-flop, the carry output CO of FA is connected to the data input port D of the D flip-flop, and the clock signal port of the D flip-flop is connected to the clock signal CLK. It should be emphasized that during the operation, the addition unit of each row will output the operation result bit by bit to the data transmission module in the order from LSB to MSB.

[0091] The data transmission module is composed of a plurality of stack units arranged in columns; each stack unit is used to receive the exponent and SUM output by the addition unit in real time in the order from low to high. i After receiving the data on each bit, it is output in reverse order to the maximum value search circuit.

[0092] The maximum number of bits supported by the stack unit in the data transmission module of this embodiment is determined by the exponent and SUM output by the addition unit. i When the exponent output by the addition unit is equal to SUM i When it is N bits, the data transmission module uses an N-bit stack unit.

[0093] In detail, Figure 4 As shown, the N-bit stack unit is composed of N D flip-flops D1 to Dn, 2n-2 tri-state buffers E1 to E2n-2, and a 2-to-1 selector MUX. The circuit connection relationship is as follows:

[0094] D1~Dn are forward cascaded through E1~En-1, the Q end of the previous stage D flip-flop is connected to the input end of the three-state buffer, and the output end of the three-state buffer is connected to the D end of the next stage D flip-flop; the control end of E1~En-1 is connected to the first enable signal EN1.

[0095] D1~Dn are also reverse cascaded through En~En-2, the D end of the next stage D flip-flop is connected to the input end of the three-state buffer, and the output end of the three-state buffer is connected to the Q end of the previous stage D flip-flop; the control end of En~En-2 is connected to the second enable signal EN2.

[0096] The clock signal ports of D1 to Dn-2 are connected to the clock signal CLK; the D end of the first-stage D flip-flop is connected to the output end of the addition unit, and is used to receive the exponents and SUM output by the addition unit in reverse order. i .

[0097] The data port 0 of the MUX is grounded, the data port 1 is connected to the Q terminal of the first-stage D flip-flop, the selection terminal is connected to the OPT signal, and the output terminal outputs the exponential and SUM arranged in sequence. i .

[0098] Specifically, Figure 4 The stack unit shown implements the bit-by-bit inversion and output of the received N-bit data, which includes two stages: data storage and data reading. The process is as follows:

[0099] (1) Data storage stage

[0100] Set EN1 to high level and EN2 to low level; and control the MUX to output the data of data port 0 through the OPT signal.

[0101] N bits of data are input from the low bit to the high bit through the D terminal of D1. After several clock cycles, the low bit to the high bit of the N bits of data are stored in Dn to D1 in sequence. At this stage, the output of MUX is 0.

[0102] (2) Data readout phase

[0103] Set EN1 to high level and EN2 to low level; and control the MUX to output the data of data port 1 through the OPT signal.

[0104] In the subsequent clock cycles, the data stored in D1 to Dn are outputted in sequence through the output port of MUX; that is, MUX outputs N-bit data arranged from high to low, realizing bit-by-bit inversion of the data input to the D end of D1 in the data storage stage.

[0105] At this point, the transmission unit has achieved bit-by-bit reversal of the exponents and SUMi calculation results output from the addition array from LSB to MSB, and outputs the exponents and calculation results arranged in order from MSB to LSB. The format of the output multi-bit number is exactly consistent with the requirement of the maximum value search circuit that each multi-bit data input should be arranged in order from MSB to LSB.

[0106] That is, the data transmission module in the exponential and normalization circuit of this embodiment realizes matching the addition array and the maximum value search circuit, and realizes orderly docking of the two major tasks of numerical addition and maximum value search required to be completed in the exponential and normalization process of floating-point MAC operation, thus realizing pipeline operation.

[0107] The maximum value search circuit in this embodiment is used to input multiple indexes and SUM i Generates a signal LOG representing the maximum index contained therein i In particular, the maximum value search circuit used in this embodiment is the circuit designed in Embodiment 1. In the maximum value search circuit, one of the input terminals of the AND gate and the input terminal of the inverter in each row of the comparison unit are directly connected to the output of the stack unit in the corresponding row.

[0108] The characteristics and working principle of the maximum value search circuit in the exponential sum normalization circuit have been described in detail in Example 1, and will not be described in detail in this embodiment. In combination with the content of Example 1, it can be seen that after the data transmission module inputs the lowest bit of the exponential sum of each row into the comparison unit of each row, the exponential sum of the corresponding row where the flag signal line LOG is still at a high level is the maximum value among all exponential sums, and it should be output as the normalization result.

[0109] Finally, the output module is used to receive the exponents and SUM output by the addition unit. i The flag signal LOG output by the comparison unit i , and the index and output corresponding to the flag signal that is still high after multiple rounds of comparison are used as the normalized result SUM.

[0110] In this embodiment, the circuit scale of the output module is adapted to the scale of the aforementioned addition array, data transmission module and maximum value search circuit. When the addition array, data transmission module and maximum value search circuit include M rows, the output module should also support the calculation of the sum of M groups of SUMs. i and LOG i Output the largest set of indices and SUM.

[0111] like Figure 5 As shown, the output module for normalizing the four groups of indexes and results includes four transmission gates arranged in columns and a cascaded OR gate; the transmission gate TG of each row i The input terminal of SUM i , the control end connects to LOG i The cascaded OR gate uses the non-zero output of each transmission gate as the final normalized result SUM.

[0112] Combined with the previous content, we can know that after exponential addition, data bit reversal and maximum value search, the SUM of each row output isi The signal flag of the row with the largest value is 1, and the signal flags of the remaining rows are all 0. Therefore, after passing through the transmission gate, only the transmission gate of the row or rows with the largest value can output the corresponding result. Finally, through the cascaded OR gate, the maximum value of the final identified exponential sum can be obtained according to the above values ​​as the normalized result SUM.

[0113] Example 3

[0114] This embodiment provides a floating-point MAC circuit for implementing multiplication and accumulation operations between multiple groups of floating-point numbers. The MAC circuit uses the exponent sum normalization circuit as in Embodiment 2 to normalize the exponent sum of each group of floating-point numbers, and outputs the maximum value of all calculated exponent sums as the normalization result.

[0115] Example 4

[0116] This embodiment further provides a CIM chip having a floating-point MAC operation function. The CIM chip integrates a floating-point MAC circuit as in the third embodiment.

[0117] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An exponential and normalizing circuit, It is characterized in that It is used to add the exponential parts of multiple groups of floating-point operands and floating-point weights in floating-point multiplication and accumulation operations, and output the maximum value of each exponential sum calculated; the exponential sum normalization circuit includes: an addition array, a data transmission module, a maximum value search circuit and an output module; The addition array includes a plurality of addition units arranged in columns; each row of the addition units is used to convert the input operand index IE i Weight index WE i Add and output the corresponding exponent and SUM i ; The data transmission module is composed of a plurality of stack units arranged in columns; each stack unit is used to receive the exponent and SUM output by the addition unit bit by bit in the order from low to high. i and output the data in reverse order to the maximum value search circuit after receiving the data in each position; The maximum value search circuit is used to input multiple exponentials and SUM i Generates a signal LOG representing the maximum index contained therein i The maximum value search circuit is composed of a plurality of comparison units arranged in columns, each comparison unit includes three NMOS tubes N1 to N3, one PMOS tube P1, one AND gate AND1, one OR gate OR1, and one inverter INV1; in each comparison unit, the source of N1 is grounded; the drain of N1 is connected to the source of N2; the drain of N2 is connected to the source of N3; the drains of N3 and P1 are connected to one of the input terminals of AND1 and serve as the output ports of the corresponding row mark signal; the source of P1 is connected to VDD; the gate of N1 is connected to the enable signal EN; The gate of P1 is connected to the PRE port; the other input end of AND1 and the input end of INV1 are connected to the output end of the stack unit of the corresponding row; the output end of AND1 is connected to one of the output ends of OR1; in the entire maximum value search circuit, the other input end of OR1 in the comparison unit of the first row is grounded, and the other input end of OR1 in the comparison unit of the remaining rows is connected to the output end of OR1 in the comparison unit of the previous row; the output end of OR1 in the comparison unit of the last row is connected to the gate of N3 in the comparison units of all rows; Output module, which is used to receive the exponents and SUM output by the addition unit i The flag signal LOG output by the comparison unit i , and the index and output corresponding to the flag signal that is still high after multiple rounds of comparison are used as the normalized result SUM.

2. The exponential and normalization circuit of claim 1, Features: Each adding unit in the adding array is composed of a serial adder FA and a D flip-flop; the two data input terminals of FA are connected to IE i WE i , the output end of FA is connected to the input end of the stack unit; the carry input end CI of FA is connected to the data output end Q of the D flip-flop, the carry output end CO of FA is connected to the data input port D of the D flip-flop, and the clock signal port of the D flip-flop is connected to the clock signal CLK.

3. The exponential and normalization circuit of claim 1, Features: The maximum number of bits supported by the stack unit in the data transfer module is determined by the exponent and SUM output by the addition unit. i Sure; When the exponent output by the addition unit is summed i When it is N bits, the data transmission module adopts an N-bit stack unit.

4. The exponential and normalization circuit of claim 3, Features: The N-bit stack unit is composed of N D flip-flops D1 to Dn, 2n-2 tri-state buffers E1 to E2n-2, and a 2-to-1 selector MUX; the circuit connection relationship is as follows: D1 to Dn are forward cascaded through E1 to En-1, the Q end of the previous D flip-flop is connected to the input end of the tri-state buffer, and the output end of the tri-state buffer is connected to the D end of the next D flip-flop; the control end of E1 to En-1 is connected to the first enable signal EN1; D1 to Dn are also reversely cascaded through En to En-2, the D end of the next D flip-flop is connected to the input end of the tri-state buffer, and the output end of the tri-state buffer is connected to the Q end of the previous D flip-flop; the control end of En to En-2 is connected to the second enable signal EN2; The clock signal ports of D1 to Dn-2 are connected to the clock signal CLK; the D end of the first-stage D flip-flop is connected to the output end of the addition unit, and is used to receive the exponents and SUM output by the addition unit in reverse order. i ; The data port 0 of the MUX is grounded, the data port 1 is connected to the Q terminal of the first-stage D flip-flop, the selection terminal is connected to the OPT signal, and the output terminal outputs the exponential and SUM arranged in sequence. i .

5. The exponential and normalization circuit of claim 4, Features: The stack unit implements the bit-by-bit inversion and output of the received N-bit data, which includes two stages: data storage and data reading. The process is as follows: (1) Data storage stage Set EN1 to high level and EN2 to low level; and control MUX to output the data of data port 0 through the OPT signal; N bits of data are input from the low bit to the high bit through the D terminal of D1. After several clock cycles, the low bit to the high bit of the N bits of data are stored in Dn to D1 in sequence. At this stage, the output of MUX is 0. (2) Data readout phase Set EN1 to high level and EN2 to low level; and control the MUX to output the data of data port 1 through the OPT signal; In the subsequent clock cycles, the data stored in D1 to Dn are outputted in sequence through the output port of MUX; that is, MUX outputs N-bit data arranged from high to low, realizing bit-by-bit inversion of the data input to the D end of D1 in the data storage stage.

6. The exponential and normalization circuit of claim 5, Features: The maximum value search circuit realizes the strategy of identifying the maximum value from the exponential sum of each N bit input and generating the corresponding flag signal as follows: (1) The respective flag signal lines LOG are connected through the PRE ports in the comparison units of each row. i Precharge to high level; (2) Receive the indexes and SUMs output by the stack cells in each row in order from high to low. i ; (3) The maximum value search circuit receives the index and SUM calculated for each row i When each bit is set, the level state of the mark signal line of each row changes and satisfies the following rules: A. If the SUM entered in each row i The corresponding bit of includes both "1" and "0", then the flag signal line of the corresponding row with data "1" maintains the previous level state, and the flag signal line of the corresponding row with data "0" becomes low level; B. When the flag signal line of any row is switched to low level, it will always remain in low level state and will no longer become high level; C. If the SUM entered in each row i If the corresponding bits of are all "1" or all "0", the mark signal lines of each row maintain the previous level state; (4) After any cycle, if only one row of the flag signal line is at a high level, the exponential sum of the input of this row is the maximum value of all exponential sums, which is taken as the normalization result; or, After completing the input of the lowest bit of the index sum of each row, the index sum of the corresponding row whose flag signal line is still at a high level is the maximum value among all the index sums, which is used as the normalization result.

7. The exponential and normalization circuit of claim 1, Features: When the addition array, data transmission module, and maximum value search circuit include M rows, the output module used by the index and normalization circuit also includes M transmission gates arranged in columns and a cascaded OR gate; the transmission gate TG of each row i The input terminal of SUM i , the control end connects to LOG i ; The cascaded OR gate uses the non-zero value output of each transmission gate as the final normalized result SUM.

8. A maximum value search circuit, Features: It is used to identify the maximum value from a plurality of multi-bit numbers input bit by bit. The maximum value search circuit is composed of a plurality of comparison units arranged in columns, each comparison unit includes three NMOS tubes N1 to N3, a PMOS tube P1, an AND gate AND1, an OR gate OR1, and an inverter INV1; in each comparison unit, the source of N1 is grounded; the drain of N1 is connected to the source of N2; the drain of N2 is connected to the source of N3; the drains of N3 and P1 are connected to one of the input terminals of AND1 and serve as the output port of the mark signal of the corresponding row; the source of P1 is connected to VDD; The gate of N1 is connected to the enable signal EN; the gate of P1 is connected to the PRE port; the other input end of AND1 is connected to the input end of INV1 and serves as the input port of each multi-bit number; the output end of AND1 is connected to one of the output ends of OR1; in the entire maximum value search circuit, the other input end of OR1 in the comparison unit of the first row is grounded, and the other input end of OR1 in the comparison unit of the remaining rows is connected to the output end of OR1 in the comparison unit of the previous row; the output end of OR1 in the comparison unit of the last row is connected to the gate of N3 in the comparison units of all rows.

9. A floating-point MAC circuit for implementing multiplication and accumulation operations between multiple groups of floating-point numbers. Features: It adopts the exponent sum normalization circuit as described in any one of claims 1 to 8 to normalize the exponent sum of each group of floating-point numbers, and outputs the maximum value of all calculated exponent sums as the normalization result.

10. A CIM chip having a floating point MAC operation function, It is characterized in that The floating-point MAC circuit as claimed in claim 9 is integrated in the CIM chip.

Citation Information

Patent Citations

  • An improved floating point multiply-adder and a floating point multiply-add calculating method

    CN107168678A

  • Storage and calculation integrated circuit device supporting floating point precision multiplication and addition operation

    CN117539827A

  • Ping-pong multiplication unit and in-memory calculation circuit and chip for reconstructing adder tree

    CN118132034A

  • Multiplication and accumulation(MAC) operator and processing-in-memory (PIM) device including the mac operator

    US20220229633A1

  • Normalizer for performing normalization and denormalization on floating-point data and operation circuit including the same

    US20230229391A1