High bit separation serial shift complement multiply-add circuit and systolic array system

Abstract

The application provides a high-bit-separated serial shift-complement multiplication-addition operation circuit and a systolic array system, which comprises: a shift-complement multiplier, which is used for performing low-bit operation processing on input operation data based on weights corresponding to different sizes of convolution kernels to obtain low-bit operation result data; and at least one high-bit-complement addition unit, which comprises a first AND gate, a first data selector, a first D flip-flop and a first full adder, the number of the high-bit-complement addition units is determined based on the bit width of the maximum intermediate result corresponding to the convolution kernel, and the at least one high-bit-complement addition unit is used for performing high-bit operation processing on the input operation data to obtain high-bit operation result data. The method provided by the application realizes expansion of different numbers of high-bit calculation channels according to the bit width of the intermediate result, and the multiplication-addition operation circuit has the advantages of simple structure, strong universality and easy expansion.

Description

Technical Field

[0001] This invention relates to the field of digital circuit design technology, and in particular to a serial shift two's complement multiply-accumulate circuit and a pulsating array system with high-order separation. Background Technology

[0002] In the circuit design process of neural network accelerators, traditional computer architecture design schemes are usually adopted. For example, the program first instructs the computation logic unit to fetch data from the register, then executes the corresponding logical operation, and then writes the operation result back to the specified register. The above operations are repeated to finally complete the operation of a program. As the amount of data processing in neural networks increases, the traditional computer architecture will bring a lot of power consumption when accessing data, and is no longer suitable for the design of current neural network accelerators.

[0003] In related technologies, the data stream is neatly stored in registers through a pulse systolic array architecture and input into the same basic operation unit (Process Element, PE) in an orderly manner. The output of each operation unit is used for the calculation of the next operation unit. Therefore, a fully pipelined calculation can be performed and the final result can be output. However, since the convolution kernels and the number of neurons of different convolutional layers are different, each convolutional layer of a conventional pulse systolic array needs to be designed with different input and output timing sequences, which makes the circuit structure design complex and lacks versatility. Summary of the Invention

[0004] This invention provides a serial shift complement multiply-accumulate circuit and pulsating array system with high-order separation, which solves the problem that the existing technology uses conventional pulse pulsating arrays, which requires different input and output timings for each layer of convolution, resulting in complex circuit structure design and lack of versatility, and improves the operation efficiency of the multiply-accumulate circuit.

[0005] This invention provides a serial shift two's complement multiply-add operation circuit with high-order bit separation, comprising:

[0006] A shift-complement multiply-accumulator is used to perform low-bit operations on the input data based on the weights corresponding to convolution kernels of different sizes, and to obtain the low-bit operation result data.

[0007] At least one high-order complement addition unit is provided, each high-order complement addition unit including a first AND gate, a first data selector, a first D flip-flop, and a first full adder. The number of high-order complement addition units is determined based on the bit width of the maximum intermediate result corresponding to the convolution kernel. The first full adder is connected to the first AND gate, the first D flip-flop, and the first data selector respectively. The first D flip-flop is also connected to the first data selector, the first data selector is also connected to the shift complement multiply-adder, and the first AND gate is also connected to an external signal input port. The at least one high-order complement addition unit is used to perform high-order operation processing on the input operation data to obtain high-order operation result data.

[0008] According to the present invention, a serial shift two's complement multiply-add operation circuit with high-order separation is provided, wherein the external signal input port includes a sign bit port and a high-order calculation start signal port;

[0009] The first AND gate includes two input terminals, one of which is connected to the sign bit port and the other of which is connected to the high-order calculation start signal port. The first AND gate is used to perform logical operations on the sign bit output from the sign bit port and the high-order calculation start signal, so that the high-order extension signal of the sign bit is obtained when the high-order calculation starts.

[0010] The first input terminal of the first full adder is connected to the output terminal of the first AND gate, and the second input terminal of the first full adder is connected to the high-order complement data output terminal.

[0011] The first data selector includes two input terminals. One input terminal is connected to the carry-out terminal of the shift two's complement multiply-adder, and the other input terminal is connected to the carry-out terminal of the first full adder. The first data selector is used to select one of the carry-out data from the output of the shift two's complement multiply-adder and the carry-out data from the output of the first full adder and output it to obtain a first selection result.

[0012] The input of the first D flip-flop is connected to the output of the first data selector, and the output of the first D flip-flop is connected to the carry input of the first full adder. The first D flip-flop is used to generate a first carry signal based on the first selection result.

[0013] The first full adder is used to add the high-order extended signal of the sign bit, the high-order complement data output from the high-order complement data output terminal, and the first carry signal to obtain the high-order operation result data.

[0014] According to the present invention, a serial shift two's complement multiply-add operation circuit with high-order separation is provided, wherein the shift two's complement multiply-adder includes:

[0015] A shift register is used to shift the input arithmetic data to obtain the shifted arithmetic data.

[0016] An inverter, the input of which is connected to the output of the shift register, is used to invert the bitwise operations of the shifted data.

[0017] The second data selector includes two input terminals. One input terminal is connected to the output terminal of the shift register, and the other input terminal is connected to the output terminal of the inverter. The second data selector is used to convert the shifted operation data from the original code to the inverse code according to the sign bit and output it to obtain the second selection result.

[0018] The second full adder has its first input connected to the output of the second data selector, its second input connected to the low-order complement data output, and its carry output connected to one input of the first data selector.

[0019] The third data selector includes two input terminals: one input terminal is connected to the sign bit port, and the other input terminal is connected to the carry output terminal of the second full adder. The third data selector is used to select one from the carry data output by the second full adder and the sign bit output by the sign bit port and output it to obtain the third selection result.

[0020] The second D flip-flop has its input connected to the output of the third data selector and its output connected to the carry input of the second full adder. The second D flip-flop is used to generate a second carry signal based on the third selection result.

[0021] The second full adder is used to add the second carry signal, the second selection result, and the low-order two's complement data output from the low-order two's complement data output terminal to obtain the low-order operation result data.

[0022] According to the present invention, a serial shift two's complement multiply-add operation circuit with high-order separation is provided, wherein the first data selector is specifically used for:

[0023] When the enable signal of the first data selector is logic 1, the output of the first data selector outputs the carry data of the second full adder.

[0024] When the enable signal of the first data selector is logic 0, the output of the first data selector outputs the carry data from the first full adder.

[0025] This invention provides a serial shift two's complement multiply-add operation circuit with high-order bit separation, wherein the second data selector is specifically used for:

[0026] When the enable signal of the second data selector is logic 1, the output of the second data selector outputs the shifted and inverted data.

[0027] When the enable signal of the second data selector is logic 0, the output of the second data selector outputs the shifted data itself.

[0028] According to the present invention, a serial shift two's complement multiply-add operation circuit with high-order separation is provided, wherein the third data selector is specifically used for:

[0029] When the enable signal of the third data selector is logic 1, the output of the third data selector outputs the carry data of the second full adder;

[0030] When the enable signal of the third data selector is logic 0, the output of the third data selector outputs the sign bit output by the sign bit port.

[0031] According to the serial shift two's complement multiply-add operation circuit with high-order separation provided by the present invention, the first data selector further includes a first control terminal, which is connected to the counter control signal output terminal. The first data selector is also used to determine the logic number of the enable terminal signal according to the counter control signal output by the counter control signal output terminal.

[0032] According to the serial shift two's complement multiply-add operation circuit with high-order separation provided by the present invention, the second data selector further includes a second control terminal, which is connected to the sign bit port. The second data selector is also used to determine the number of logic bits of the enable signal based on the sign bit output by the sign bit port.

[0033] According to the high-bit separation serial shift two's complement multiply-add operation circuit provided by the present invention, the third data selector further includes a third control terminal, which is connected to the addend availability signal output terminal. The third data selector is also used to determine the logic number of the enable terminal signal based on the addend availability signal output by the addend availability signal output terminal.

[0034] The present invention also provides a pulsating array system, comprising:

[0035] Multiple high-order separated serial shift two's complement multiplication and addition circuits.

[0036] The serial shift-two's complement multiply-add circuit and pulsating array system provided by this invention perform low-bit operations on the input data according to the weights corresponding to different convolution kernels through a shift-two's complement multiply-adder to obtain low-bit operation result data. On this basis, at least one high-bit two's complement addition unit is used to obtain high-bit operation result data. This realizes the expansion of different numbers of high-bit calculation paths according to the bit width of the intermediate result, ensuring that in pulsating arrays with different structures, adjacent input data are serially input into the pulse pulsating array at fixed intervals for pipelined calculation. There is no need to adjust the interval of the input data and the storage form of the data in the memory according to the bit width of the intermediate part. Moreover, the multiply-add circuit has the advantages of simple structure, strong versatility and easy expansion. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is one of the schematic diagrams of the serial shift two's complement multiplication and addition operation circuit with high-order separation provided by the present invention;

[0039] Figure 2 This is the second schematic diagram of the high-bit separation serial shift two's complement multiplication and addition operation circuit provided by the present invention;

[0040] Figure 3 This is the third schematic diagram of the high-bit separation serial shift two's complement multiplication and addition operation circuit provided by the present invention;

[0041] Figure 4 This is a timing diagram of the serial shift two's complement multiplication and addition operation circuit with high-order separation provided by the present invention;

[0042] Figure 5 This is one of the structural schematic diagrams of the pulsating array system provided by the present invention;

[0043] Figure 6 This is the second schematic diagram of the pulsating array system provided by the present invention.

[0044] Figure label:

[0045] 110: High-order complement addition unit; 111: First AND gate;

[0046] 112: First data selector; 113: First D flip-flop; 114: First full adder;

[0047] 120: Shift-two's complement multiply-adder; 121: Shift register; 122: Inverter;

[0048] 123: Second data selector; 124: Second full adder;

[0049] 125: Third data selector; 126: Second D flip-flop. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] The following is combined with Figures 1-5 The present invention describes a serial shift two's complement multiply-accumulate circuit and a pulsating array system with high-order separation.

[0052] Figure 1 This is one of the structural schematic diagrams of the serial shift two's complement multiplication and addition operation circuit with high-order separation provided by the present invention, such as... Figure 1 As shown, the serial shift two's complement multiply-add circuit 100 with high-order separation includes: at least one high-order two's complement adder 110, a first AND gate 111, a first data selector 112, a first D flip-flop 113, a first full adder 114, and a shift two's complement multiply-adder 120.

[0053] The first full adder 114 is connected to the first AND gate 111, the first D flip-flop 113 and the first data selector 112 respectively. The first D flip-flop 113 is also connected to the first data selector 112. The first data selector 112 is also connected to the shift complement multiply-adder 120. The first AND gate 111 is also connected to the external signal input port. At least one high-order complement addition unit 110 is used to perform high-order operation processing on the input operation data to obtain the high-order operation result data.

[0054] In this embodiment, the data selector can be a 2-to-1 data selector, which includes two inputs and one output. The sign bit of different input signals is determined according to the control terminal corresponding to each data selector, and the output signal is determined according to the number of logic bits of the sign bit.

[0055] The number of high-order complement addition units is determined based on the bit width of the maximum intermediate result corresponding to the convolution kernel.

[0056] In this embodiment, the number of high-order complement addition units 110 set in each PE is determined based on the bit width of the maximum intermediate result corresponding to the convolution kernel.

[0057] It should be noted that the minimum interval between data inputs in a systolic array architecture is affected by the number of preprocessor (PE). For example, for a circuit design with 8-bit wide activation values ​​and 3-bit wide weights, the input data becomes 16 bits after shifting. Therefore, the input interval for the activation value of the same PE must be at least 16 clock cycles to ensure that the output does not conflict. For a 3×3×64 convolution kernel (with a kernel length of 3, a width of 3, and 64 channels), each row of the systolic array requires 576 PEs during convolution. The largest intermediate result is 26 bits wide. Therefore, the input data of the same PE must be spaced at least 26 clock cycles apart to ensure that the systolic calculation does not conflict. During the calculation of the PE, the shift register 121 has completed the calculation in the first 16 clock cycles, and only the full adder runs in the last 10 clock cycles, which wastes a lot of computing resources.

[0058] In this embodiment, for convolution kernels of different sizes, the maximum number of bits in the corresponding intermediate result is different, and the interval of the data input for the same PE will also change. The high-bit two's complement addition unit 110 can meet the requirements of high-bit multiplication and addition operations.

[0059] In this embodiment, when the bit width of the intermediate result is 26 bits, it includes two parts: the lower 16 bits and the higher 10 bits. During the systolic array calculation process, the first 16 bits of data are calculated by the shift complement multiplier-adder 120, and then the higher bits of data (e.g., 16 bits to 32 bits, 33 bits to 48 bits, etc.) are calculated by a higher complement addition unit 110 extended by each PE. The two calculation paths of a single PE are continuous in time and together complete the calculation of the 26-bit value.

[0060] The serial shift-two's complement multiply-add circuit of the present invention, which separates high-order bits, performs low-order operations on the input operation data according to the weights corresponding to different convolution kernels by using a shift-two's complement multiply-adder. On this basis, it obtains the high-order operation result data through at least one high-order two's complement addition unit. This realizes the expansion of different numbers of high-order calculation paths according to the bit width of the intermediate result. The multiply-add circuit has the advantages of simple structure, strong versatility and easy expansion.

[0061] In some embodiments, the external signal input port includes a sign bit port and a high-order calculation start signal port.

[0062] In this embodiment, the first AND gate 111 includes two input terminals, one of which is connected to the sign bit port and the other of which is connected to the high-order calculation port. The first AND gate 111 is used to perform logical operations on the sign bit and the high-order calculation signal output by the high-order calculation port to obtain the high-order extended signal of the sign bit.

[0063] exist Figure 1 In the embodiment shown, the first AND gate 111 includes AND gates 1 to N, each of which is identical. One input of AND gate 1 is connected to the sign bit port SignBit, and the other input is connected to the high-order bit calculation port SH_Start_1.

[0064] In this embodiment, the first input terminal of the first full adder 114 is connected to the output terminal of the first AND gate 111, and the second input terminal of the first full adder 114 is connected to the high-order complement data output terminal.

[0065] exist Figure 1 In the embodiment shown, the first full adder 114 includes full adders 2 to N+1, and each full adder is identical. The first input terminal A of full adder 2 is connected to the output terminal of the first AND gate 111, and the second input terminal B of full adder 2 is connected to the high-order complement data output terminal Pre_Sum_H_1.

[0066] The first data selector 112 includes two input terminals. One input terminal is connected to the output terminal of the second full adder 124, and the other input terminal is connected to the carry output terminal of the first full adder 114. The first data selector 112 is used to select one of the carry data from the output of the second full adder 124 and the carry data from the output of the first full adder 114 and output it to obtain the first selection result.

[0067] exist Figure 1 In the illustrated embodiment, the first data selector 112 includes MUX3 to MUXN+2, and each data selector is identical. MUX3 includes two input terminals, one of which is connected to the output terminal of the second full adder 124 (full adder 1), and the other input terminal is connected to the carry output terminal of the first full adder 114 (full adder 2). The first data selector 112 is used to select one of the carry data from the output of full adder 1 and the carry data from the output of full adder 2 and output it to obtain the first selection result.

[0068] In this embodiment, the two inputs of the first data selector (MUX3) receive the carry output signal of the second full adder 124 and the carry output signal of the first full adder 114, respectively.

[0069] In this embodiment, the first data selector further includes a first control terminal, which is connected to the counter control signal output terminal. The first data selector is also used to determine the logic number of the enable terminal signal based on the counter control signal output by the counter control signal output terminal.

[0070] In this embodiment, when the enable signal of the first data selector is logic 1, the output of the first data selector outputs the carry data of the second full adder; when the enable signal of the first data selector is logic 0, the output of the first data selector outputs the carry data of the first full adder.

[0071] The input of the first D flip-flop 113 is connected to the output of the first data selector 112, and the output of the first D flip-flop 113 is connected to the carry input of the first full adder 114. The first D flip-flop 113 is used to generate a first carry signal according to the first selection result. The first full adder 114 is used to add the high-order extension signal of the sign bit, the high-order complement data output from the high-order complement data output terminal, and the first carry signal to obtain the high-order operation result data.

[0072] exist Figure 1 In the illustrated embodiment, the first D flip-flop 113 includes D flip-flops 2 to N+2, all of which are identical. The input of D flip-flop 2 is connected to the output of MUX3, and the output of D flip-flop 2 is connected to the carry input of full adder 2. D flip-flop 2 is used to generate a first carry signal based on the first selection result. Full adder 2 is used to add the high-order extension signal of the sign bit, the high-order two's complement data output from the high-order two's complement data output terminal, and the first carry signal to obtain the high-order operation result data.

[0073] In this embodiment, the number of high-order complement addition units 110 in the PE may be adjusted according to the bit width of the intermediate result of the convolution kernel of the neural network. For example, when the intermediate result is between 33 bits and 48 bits, the first 16 bits are calculated by the shift complement multiplier-adder 120. Based on this, the first high-order complement addition unit 110 is introduced to complete the calculation of the 17th to 32nd bits, and the second high-order complement addition unit 110 is introduced to complete the calculation of the 33rd to 48th bits, until all high-order bits are calculated. Finally, the low-order operation result data and the high-order operation result data are combined to obtain the final output result.

[0074] The serial shift two's complement multiplication and addition circuit with high-order separation in this embodiment of the invention can obtain the high-order operation result data of the input signal by logically connecting each unit of the high-order two's complement addition unit, and realizes the expansion of different numbers of high-order calculation paths according to the bit width of the intermediate result.

[0075] In some embodiments, the shift complement multiply-adder 120 includes: a shift register 121, an inverter 122, a second data selector 123, a second full adder 124, a third data selector 125, and a second D flip-flop 126.

[0076] The shift register 121 is used to shift the input computation data based on the weights corresponding to convolution kernels of different sizes, so as to obtain the shifted computation data.

[0077] In this embodiment, the multiplication operation between the input data and the weight is implemented. The shift register 121 can be implemented in various ways, including but not limited to using D flip-flops and a look-up table (LUT).

[0078] In this embodiment, the convolution kernel size of different neural networks may be different, and the weights corresponding to convolution kernels of different sizes are different, so the weights allocated for shifting the input data are different.

[0079] In this embodiment, in the neural network, the output of the previous layer serves as the input activation value for the next layer. The signal in the network is a pulse signal, and different ports can output serial signals in parallel. For example, the serial signal is data i_act, and different bit lengths correspond to different input ports of the PE. Different i_act values ​​are updated at different times to participate in the multiplication and addition operations of the PE. Each PE completes the multiplication of one input data with its weight and the addition of the sum of the input parts. The weight W represents an integer power of 2 (2^32). W The index value of ).

[0080] It should be noted that before the weights are deployed to the hardware, the storage format of the weights needs to be changed. When storing the weights, only the exponential value of the weight is saved. Each PE can use shift register 121 to process the data according to the weight value with different delays to realize the synaptic weighting process. The shift register 121 in the PE is a delay circuit. Specifically, when performing a multiplication operation, the input data and output data are delayed by different clock cycles according to the weight.

[0081] The input of inverter 122 is connected to the output of shift register 121. Inverter 122 is used to invert the shifted data bit by bit.

[0082] In this embodiment, after the input arithmetic data is weighted and output by the delay circuit, the sign needs to be determined and converted into the inverse code before the accumulation calculation in PE can be performed. The shifted arithmetic data can be converted into the corresponding inverse code by the inverter 122, and then added to the sign bit in the first calculation cycle of the full adder 124 to obtain the complement code corresponding to the original input data.

[0083] The second data selector 123 includes two input terminals. One input terminal is connected to the output terminal of the shift register 121, and the other input terminal is connected to the output terminal of the inverter 122. The second data selector 123 is used to convert the shifted operation data from the original code to the inverse code according to the sign bit and output it to obtain the first selection result.

[0084] exist Figure 1 In the embodiment shown, the second data selector 123 is MUX1, one input of MUX1 is connected to the output of shift register 121, and the other input is connected to the output of inverter 122.

[0085] In this embodiment, the second control terminal of the second data selector 123 (MUX1) is connected to the sign bit, and the two input terminals are respectively connected to the delayed output data (the shifted arithmetic data) and the inverted delayed output data (the inverted data corresponding to the shifted arithmetic data).

[0086] The serial shift-two's complement multiply-add circuit with high-order separation in this embodiment of the invention realizes low-order operation on the input signal through the shift-two's complement multiply-adder, and combines the low-order operation result data with the high-order operation result data output by the high-order complement addition unit to arbitrarily input the corresponding operation result.

[0087] In some embodiments, the first data selector 112 further includes a first control terminal, which is connected to the counter control signal output terminal. The first data selector 112 is also used to determine the logic number of the enable terminal signal based on the counter control signal output by the counter control signal output terminal.

[0088] In this embodiment, the specific steps for determining the logic bit length of the enable signal of the first data selector 112 (MUX3) are as follows:

[0089] (1) When the enable signal of the first data selector 112 is logic 1, the output of the first data selector 112 outputs the carry data of the second full adder 124.

[0090] (2) When the enable signal of the first data selector 112 is logic 0, the output terminal of the first data selector 112 outputs the carry data of the first full adder 114.

[0091] In some embodiments, the second data selector 123 is specifically used to: when the enable signal of the second data selector 123 is logic 1, the output of the second data selector 123 outputs the bit-inverted data of the shifted arithmetic data; when the enable signal of the second data selector 123 is logic 0, the output of the second data selector 123 outputs the inverse code of the shifted positive arithmetic data.

[0092] For example, when the sign bit of MUX1 is 0, the weight is positive, MUX1 selects channel 0, and the output data is the delayed data itself, that is, the inverse code of the positive number; when the sign bit is 1, the weight is negative, MUX1 selects channel 1, and the output data is delayed and then passes through inverter 122; after passing through MUX1, the delayed data becomes the inverse code of the negative number and is sent to the first input terminal (port A) of the second full adder 124.

[0093] In this embodiment, the second data selector 123 further includes a second control terminal, which is connected to the sign bit port. The second data selector 123 is also used to determine the number of logic bits of the enable signal based on the sign bit output by the sign bit port.

[0094] In this embodiment, the specific steps for determining the number of logic bits of the enable signal of MUX1 are as follows:

[0095] (1) When the enable signal of the second data selector 123 is logic 1, the output of the second data selector 123 outputs the inverse code of the shifted negative number operation data.

[0096] (2) When the enable signal of the second data selector 123 is logic 0, the output of the second data selector 123 outputs the inverse code of the shifted positive number.

[0097] The first input terminal of the second full adder 124 is connected to the output terminal of the second data selector 123, and the second input terminal of the second full adder 124 is connected to the low-order complement data output terminal.

[0098] In this embodiment, the second input port (port B) of the second full adder 124 receives the low-order data of the input data, and the second full adder 124 performs addition operations on the input data and outputs operation result data with different bit widths; for example, the second full adder 124 outputs the low-order operation result corresponding to a bit width of 0-16 bits.

[0099] The third data selector 125 includes two input terminals. One input terminal is connected to the sign bit port, and the other input terminal is connected to the carry output terminal of the second full adder 124. The output terminal of the third data selector 125 is connected to the input terminal of the second D flip-flop 126. The third data selector 125 is used to select one of the carry data output from the second full adder 124 and the sign bit output from the sign bit port and output it to obtain the third selection result.

[0100] In this embodiment, the two inputs of the third data selector 125 (MUX2) receive the sign bit and the carry output signal of the second full adder 124, respectively, that is, output carry data.

[0101] In this embodiment, the third data selector 125 further includes a third control terminal, which is connected to the addend available signal output terminal. The third selector 125 is also used to determine the logic number of the enable terminal signal based on the addend available signal output by the addend available signal output terminal.

[0102] In this embodiment, the specific steps for determining the logic bit length of the enable signal of the third data selector 125 (MUX2) are as follows:

[0103] (1) When the enable signal of the third data selector 125 is logic 1, the output of the third data selector 125 outputs the carry data of the second full adder 124.

[0104] (2) When the enable signal of the third data selector 125 is logic 0, the output terminal of the third data selector 125 outputs the sign bit of the sign bit output by the sign bit port.

[0105] In this embodiment, when the third control terminal signal is 0, channel 0 is selected, the output signal is the sign bit, and the output is buffered in the second D flip-flop 126; when the third control terminal is 1, channel 1 is selected, the output signal is the carry output of the second full adder 124, and the output is buffered in the second D flip-flop 126.

[0106] The output of the second D flip-flop 126 is connected to the carry input of the second full adder 124. The second D flip-flop 124 is used to generate a second carry signal based on the third selection result.

[0107] In this embodiment, the second D flip-flop 126 is used to latch the carry input (via the Ci port input) of the second full adder 124.

[0108] The second full adder 124 is used to add the second carry signal, the second selection result, and the low-order two's complement data output from the low-order two's complement data output terminal to obtain the low-order operation result data.

[0109] In this embodiment, the A port of the second full adder 124 receives the inverse code of the delayed data, the B port receives the low-order complement code of the convolutional sum, and the Ci port receives the sign bit or the carry output of the second full adder 124 from the previous clock cycle. The first clock cycle is calculated, and the carry input of the second full adder 124 is the sign bit. If the sign bit is 0, the input value at the A port of the full adder is a positive number, and its complement code is itself. If the sign bit is 1, the input value at the A port of the full adder is the inverse code obtained by inverting the original code. The first cycle is added to the sign value 1 to obtain the complement code, thus realizing the conversion from the original code to the complement code. The low-order operation result data (Sum_L) output by the second full adder 124 is the low-order data of the convolutional sum.

[0110] In this embodiment, the PE performs power-of-two multiplication through shift register 121, converts sign-magnitude representation to complement representation through inverter 122 and second data selector 123, obtains sign bit data and output carry data through third data selector 125 and second D flip-flop 126, and performs addition through second full adder 124. This enables power-of-two multiplication and accumulation operations with both positive and negative values, reducing hardware resource overhead and improving computational efficiency.

[0111] In this embodiment, when the bit width of the intermediate result processed by the convolution kernel is 33 bits, all the high-order bits can be calculated by adding two high-order complement addition units 170 to each PE. It should be noted that no matter how many high-order complement addition units 110 are added, after the low-order 16 bits are calculated, the low-order calculation path can start calculating the low-order 16 bits of the next input data. The input activation value of each PE always maintains an interval of 16 clock cycles. Therefore, the systolic array architecture including the high-order complement addition unit 110 has good versatility.

[0112] In this embodiment, when each PE outputs its calculated data, the first 16 clock cycles output the calculation result of the lower 16 bits from the low bit to the high bit through the first addition path (shift complement multiplier-adder 120); from the 17th to the 32nd clock cycles, the calculation result of the 17th to the 32nd bits is output from the second addition path (a high-order complement adder 110); if the calculation of the current convolution kernel is not completed, the outputs of the two addition paths are connected to the inputs of the two addition paths of the next PE; if the calculation of the convolution kernel is completed, the output data of the two addition paths can be concatenated to obtain the complete convolution calculation result.

[0113] Figure 2 This is the second schematic diagram of the high-bit separation serial shift two's complement multiplication and addition operation circuit provided by the present invention. Figure 2In the illustrated embodiment, a high-order complement addition unit 110 is extended in each PE, including: AND gate 1 (same as the first AND gate 111), full adder 2, D flip-flop 2, and MUX3, to complete the calculation of the 17th to 32nd bits of the intermediate result; wherein, MUX3 is connected to the counter control signal CntCtrl, and the two inputs of MUX3 receive the carry output of full adder 1 and the carry output of full adder 2 respectively, and MUX3 is used to select the carry input of full adder 2; the inputs of AND gate 1 are connected to the highest-order calculation start signal SH_Start and the sign bit signal SignBit respectively, and the high-order extension of the output sign bit is connected to the A input port of full adder 2; D flip-flop 2 is used to latch the carry input of full adder 2; the inputs of full adder 2 are the output of AND gate 1, the high-order complement data Pre_Sum_H input to the module, and the carry signal output by D flip-flop 2 respectively; the full adder 1 of this module outputs the lower 16 bits of the calculation result, and the full adder 2 outputs the second highest 16 bits of the calculation result.

[0114] Figure 3 This is the third schematic diagram of the high-bit separation serial shift two's complement multiplication and addition operation circuit provided by the present invention. Figure 3 In the illustrated embodiment, an additional high-order complement addition unit 110 is added to each PE, including: AND gate 2 (same as the first AND gate), full adder 3, D flip-flop 3, and MUX4, to complete the calculation of bits 33 to 48 of the intermediate result; wherein, MUX4 is connected to the counter control signal CntCtrl_2, and the two inputs of MUX4 receive the carry output of full adder 2 and the carry output of full adder 3 respectively, and MUX4 is used to select the carry input of full adder 3; the inputs of AND gate 2 are respectively connected to the start of calculation of the most significant bit. The signal SH_Start_2 and the sign bit signal SignBit, with the high-order extension of the output sign bit connected to the A input port of full adder 3; D flip-flop 3 is used to latch the carry input of full adder 3; the inputs of full adder 3 are the output of AND gate 2, the high-order complement data Pre_Sum_H2 input to the module, and the carry signal output by D flip-flop 3, respectively; full adder 1 of this module outputs the lower 16 bits of the calculation result, full adder 2 outputs the second highest 16 bits of the calculation result, and full adder 3 outputs the highest bit of the calculation result.

[0115] Figure 4 This is a timing diagram of the serial shift two's complement multiplication and addition circuit with high-order separation provided by the present invention. Figure 4In the illustrated embodiment, for a single PE, the 8-bit input data is: i_act = 8'b01100100, the 3-bit weight is: W = 3'b011, the sign bit SignBit = 1, Pre_Sum_L = 16'h0002, Pre_Sum_H = 16'h0001, resulting in a delay period that can be represented by the weight from 0 to 7. Specifically, the weight W = 3'b011 represents a delay period of 2. 3 The delay circuit requires a delay of three clock cycles, so the output data is 11'b01100100000, with the least significant bit (LSB) being output first. Since the input data is delayed by a maximum of seven clock cycles, the data after shifting is at most 15 bits wide, reserving one bit for carry, resulting in a 16-bit wide output. With the sign bit set to 1, the data from MUX1 output channel 1 is 16'1111110011011111. After one cycle of data input, the shifted signal is output, and the partial sum usable signal Sum_vld goes high. The input full adder 1 is added to the lower 16 bits of the convolution partial sum data Pre_Sum_L = 16'h0002, with the carry signal during the first bit addition being the sign bit 1. After the last cycle of low-bit calculation, the Sum_vld signal is turned off, latching the sign bit into the first D flip-flop 160, ready to participate in the calculation of the next data.

[0116] In this embodiment, after the low-order bit calculation is completed, the high-order bit calculation signal SH_Start goes high to begin high-order bit calculation, and the sign extension is added to Pre_Sum_H = 16'h0001; simultaneously, the counter control signal CntCtrl is set to high and held for one clock cycle, controlling MUX3 to intercept the carry signal of the 16th bit of the low-order bit calculation for high-order bit calculation; the output data timing is as follows: the low 16 bits of data are output first from Sum_L, then after 16 cycles, Sum_L outputs the low 16 bits after the second activation value calculation, while the first activation value output by Sum_H is processed by the neural network convolution to calculate the high 16 bits of the data, obtaining the final result. If Sum_L = 16'hfce2 and Sum_H = 16'h0000, then convert them to decimal numbers for verification. The input activation value i_act = 8'b01100100 = 100; the weight W = 3'b011 = 3, and the sign bit SignBit = 1. Therefore, the output data after shifting is -800. The input part and the lower 16 bits are Pre_Sum_L = 16'h0002, and the higher 16 bits are Pre_Sum_H = 16'h0001, totaling 32'h00010002 = 65538. The final result is 65538 - 800 = 64738 = 32'0000hfce2.

[0117] In this embodiment, while the PE performs calculations on the high-order bits, the low-order addition path simultaneously calculates the lower 16 bits of the second data. The input activation value is 1, and the shifted data is -8. The lower 16 bits of the partial sum are input as 16'h0010 = 16, and the final calculation result of the lower 16 bits is 16'h0008. Thus, the calculation of all 32 bits of the first data and the calculation of the lower 16 bits of the second data are both correct, and the PE can calculate the high 16 bits of the first data and the lower 16 bits of the second data simultaneously, thereby ensuring that the PE can achieve the expected function.

[0118] In this embodiment, for a 3×3×64 convolution kernel, 576 PEs need to be connected together to complete the calculation. With an 8-bit activation value and a 3-bit weight width, the maximum convolution sum may be 26 bits wide, where the highest bit is the sign bit, resulting in a 32-bit wide output, where the high 6 bits are the sign extension. For convolution kernels of different sizes, such as a 3×3×128 kernel, the maximum convolution sum requires a 27-bit width. PEs of this structure can be directly connected, and the data can maintain the original input mode, resulting in 32-bit wide data, where the high 5 bits are the sign extension. At the same time, for convolution sums with higher bit widths, different numbers of high-bit modules can be extended in the PE according to the bit width. This serial shift complement multiply-add circuit with high-bit separation can realize the calculation of convolution kernels of different sizes, and the overall circuit design structure is relatively simple, highly versatile, and easy to expand.

[0119] The serial shift-two's complement multiply-add circuit of this invention, which separates high-order bits, performs low-order operations on the input data according to the weights corresponding to different convolution kernels through a shift-two's complement multiply-adder. On this basis, it obtains the high-order operation result data through at least one high-order complement addition unit. This realizes the expansion of different numbers of high-order calculation paths according to the bit width of the intermediate result, ensuring that in pulse arrays with different structures, adjacent input data are serially input into the pulse array at fixed intervals for pipelined calculation. There is no need to adjust the interval of the input data and the storage form of the data in the memory according to the bit width of the intermediate part. Moreover, the multiply-add circuit has the advantages of simple structure, strong versatility and easy expansion.

[0120] The pulsating array system provided by the present invention is described below. The pulsating array system described below can be referred to in correspondence with the serial shift two's complement multiplication and addition circuit with high-order separation described above.

[0121] Figure 5 This is one of the structural schematic diagrams of the pulsating array system provided by the present invention, such as... Figure 5 As shown, the pulsating array system includes multiple serial shift complement multiply-add operation circuits 100 with high-order separation.

[0122] Each serial shift two's complement multiply-add circuit with high-order separation includes: a shift two's complement multiply-adder 120 and at least one high-order two's complement adder 110.

[0123] Shift registers are used to shift input computational data based on the weights corresponding to convolution kernels of different sizes, to obtain shifted computational data.

[0124] In this embodiment, the convolution kernel size of different neural networks may be different, and the weights corresponding to convolution kernels of different sizes are different, so the weights allocated for shifting the input data are different.

[0125] The input of the inverter is connected to the output of the shift register. The inverter is used to invert the original code of the shifted data.

[0126] In this embodiment, after the input computational data is weighted and output by the delay circuit, the sign needs to be determined and converted into two's complement before the accumulation calculation in PE can be performed. The shifted computational data can be converted into the corresponding one's complement by an inverter and a second data selector.

[0127] The second data selector includes two input terminals. One input terminal is connected to the output terminal of the shift register, and the other input terminal is connected to the output terminal of the inverter. The second data selector is used to convert the shifted data from its original code to its inverse code and output it to obtain the second selection result.

[0128] In this embodiment, the second control terminal of the second data selector (MUX1) is connected to the sign bit, and the two input terminals are respectively connected to the delayed output data (the shifted arithmetic data) and the inverted delayed output data (the inverted data after shifting arithmetic data).

[0129] In this embodiment, the second data selector further includes a second control terminal, which is connected to the sign bit port. The second data selector is also used to determine the number of logic bits of the enable signal based on the sign bit output by the sign bit port.

[0130] In this embodiment, when the enable signal of the second data selector is logic 1, the output of the second data selector outputs the inverse code of the shifted negative number operation data; when the enable signal of the second data selector is logic 0, the output of the second data selector outputs the inverse code of the shifted positive number operation data.

[0131] The first input of the second full adder is connected to the output of the second data selector, and the second input of the second full adder is connected to the low-order complement data output.

[0132] The third data selector includes two input terminals: one input terminal is connected to the sign bit port, and the other input terminal is connected to the carry output terminal of the second full adder. The output terminal of the third data selector is connected to the input terminal of the second D flip-flop. The third data selector is used to select one of the carry data output from the second full adder and the sign bit output from the sign bit port and output it to obtain the third selection result.

[0133] In this embodiment, the two inputs of the third data selector (MUX2) receive the sign bit and the carry output signal of the second full adder, respectively, and thus output carry data.

[0134] In this embodiment, the third data selector further includes a third control terminal, which is connected to the addend availability signal output terminal. The third data selector is also used to determine the logic bit length of the enable terminal signal based on the addend availability signal output by the addend availability signal output terminal.

[0135] In this embodiment, when the enable signal of the third data selector is logic 1, the output of the third data selector outputs the carry data from the second full adder; when the enable signal of the third data selector is logic 0, the output of the third data selector outputs the sign bit from the sign bit port.

[0136] The output of the second D flip-flop is connected to the carry input of the second full adder. The second D flip-flop is used to generate the second carry signal based on the third selection result.

[0137] In this embodiment, the second D flip-flop is used to latch the carry input (via the Ci port input) of the second full adder.

[0138] The second full adder is used to add the second carry signal, the second selection result, and the low-order two's complement data output from the low-order two's complement data output terminal to obtain the low-order operation result data.

[0139] In this embodiment, the A port of the second full adder receives the inverse code of the delayed data, the B port receives the low-order complement code of the convolutional sum, and the Ci port receives the sign bit or the carry output of the first full adder from the previous clock cycle. The first clock cycle is calculated, and the carry input of the second full adder is the sign bit. If the sign bit is 0, the input value at the A port of the full adder is a positive number, and its complement code is itself. If the sign bit is 1, the input value at the A port of the full adder is the inverse code obtained by inverting the original code. The complement code is obtained by adding the sign value 1 in the first cycle, thus realizing the conversion from the original code to the complement code. The low-order operation result data (Sum_L) output by the second full adder is the low-order data of the convolutional sum.

[0140] At least one high-order complement addition unit, each high-order complement addition unit including a first AND gate, a first data selector, a first D flip-flop and a first full adder, the number of high-order complement addition units being determined based on the bit width of the maximum intermediate result corresponding to the convolution kernel.

[0141] In this embodiment, for convolution kernels of different sizes, the maximum number of bits in the corresponding intermediate result is different, and the interval of the data input for the same PE will also change. The high-bit two's complement addition unit can meet the requirements of high-bit multiplication and addition operations.

[0142] For example, when the bit width of the intermediate result is 26 bits, it includes two parts: the lower 16 bits and the higher 10 bits. During the systolic array calculation process, the first 16 bits of data are calculated by a shift two's complement multiply-adder composed of a shift register, an inverter, a second data selector, a second full adder, a third data selector, and a second D flip-flop. Then, the higher bits of data (e.g., 16 bits to 32 bits, 33 bits to 48 bits, etc.) are calculated by a higher two's complement addition unit extended by each PE. The two calculation paths of a single PE are continuous in time and together complete the calculation of the 26-bit value.

[0143] In this embodiment, the first AND gate includes two input terminals, one of which is connected to the sign bit port and the other of which is connected to the high-order calculation port. The first AND gate is used to perform logical operations on the sign bit and the high-order calculation signal output by the high-order calculation port to obtain the high-order extended signal of the sign bit.

[0144] In this embodiment, the first input of the first full adder is connected to the output of the first AND gate, and the second input of the first full adder is connected to the high-order complement data output.

[0145] The first data selector includes two input terminals. One input terminal is connected to the output terminal of the second full adder, and the other input terminal is connected to the carry output terminal of the first full adder. The first data selector is used to select one of the carry data from the output of the second full adder and the carry data from the output of the first full adder and output it to obtain the first selection result.

[0146] In this embodiment, the two inputs of the first data selector (MUX3) receive the carry output signal of the second full adder and the carry output signal of the first full adder, respectively.

[0147] In this embodiment, the first data selector further includes a first control terminal, which is connected to the counter control signal output terminal. The first data selector is also used to determine the logic number of the enable terminal signal based on the counter control signal output by the counter control signal output terminal.

[0148] In this embodiment, when the enable signal of the first data selector is logic 1, the output of the first data selector outputs the carry data of the second full adder; when the enable signal of the first data selector is logic 0, the output of the first data selector outputs the carry data of the first full adder.

[0149] The input of the first D flip-flop is connected to the output of the first data selector, and the output of the first D flip-flop is connected to the carry input of the first full adder. The first D flip-flop is used to generate a first carry signal based on the first selection result. The first full adder is used to add the high-order extension signal of the sign bit, the high-order complement data output from the high-order complement data output terminal, and the first carry signal to obtain the high-order operation result data.

[0150] In this embodiment, the number of high-order complement addition units in the PE may be adjusted according to the bit width of the intermediate result of the convolution kernel of the neural network. For example, when the intermediate result is between 33 bits and 48 bits, the first 16 bits are calculated by a shift complement multiplier-adder. Based on this, the first high-order complement addition unit is introduced to complete the calculation of the 17th to 32nd bits, and the second high-order complement addition unit is introduced to complete the calculation of the 33rd to 48th bits, until all high-order bits are calculated. Finally, the low-order operation result data and the high-order operation result data are combined to obtain the final output result.

[0151] Figure 6 This is a second schematic diagram of the pulsating array system provided by the present invention, as shown below. Figure 6 As shown, the systolic array system includes multiple serial shift complement multiply-add operation circuits 100 with high-order separation. The systolic array system includes n rows and M columns. The input data flows in the systolic array system, reducing the number of memory accesses and making the structure more regular and the wiring more uniform. Among them, a[1]-a[M] represents M input signals.

[0152] The pulsating array system of this invention uses multiple serial shift-two's complement multiply-add circuits with high-order separation in series. Each multiply-add circuit's shift-two's complement multiply-adder performs low-order operations on the input data according to the weights corresponding to different convolution kernels. Based on this, at least one high-order complement addition unit is used to obtain the high-order operation result data. This realizes the expansion of different numbers of high-order calculation paths according to the bit width of the intermediate result, ensuring that in pulsating arrays with different structures, adjacent input data are serially input into the pulse pulsating array at fixed intervals for pipelined calculation. There is no need to adjust the interval of the input data and the storage format of the data in the memory according to the bit width of the intermediate part. Moreover, the multiply-add circuit has the characteristics of simple structure, strong versatility, and easy expansion, thereby improving the versatility and scalability of the pulsating array system.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A serial shift two's complement multiplication and addition circuit with high-order separation, characterized in that, include: A shift-complement multiply-accumulator is used to perform low-bit operations on the input data based on the weights corresponding to convolution kernels of different sizes, and obtain the low-bit operation result data. At least one high-order complement addition unit is provided, each high-order complement addition unit including a first AND gate, a first data selector, a first D flip-flop, and a first full adder. The number of high-order complement addition units is determined based on the bit width of the maximum intermediate result corresponding to the convolution kernel. The first full adder is connected to the first AND gate, the first D flip-flop, and the first data selector respectively. The first D flip-flop is also connected to the first data selector. The first data selector is also connected to the shift complement multiply-adder. The first AND gate is also connected to an external signal input port. The at least one high-order complement addition unit is used to perform high-order operation processing on the input operation data to obtain high-order operation result data. The shift-two's complement multiplier-adder includes: A shift register is used to shift the input arithmetic data to obtain the shifted arithmetic data. An inverter, the input of which is connected to the output of the shift register, is used to invert the bitwise operations of the shifted data. The second data selector includes two input terminals. One input terminal is connected to the output terminal of the shift register, and the other input terminal is connected to the output terminal of the inverter. The second data selector is used to convert the shifted operation data from the original code to the inverse code according to the sign bit and output it to obtain the second selection result. The second full adder has its first input connected to the output of the second data selector, its second input connected to the low-order complement data output, and its carry output connected to one input of the first data selector. The third data selector includes two input terminals: one input terminal is connected to the sign bit port, and the other input terminal is connected to the carry output terminal of the second full adder. The third data selector is used to select one from the carry data output by the second full adder and the sign bit output by the sign bit port and output it to obtain the third selection result. The second D flip-flop has its input connected to the output of the third data selector and its output connected to the carry input of the second full adder. The second D flip-flop is used to generate a second carry signal based on the third selection result. The second full adder is used to add the second carry signal, the second selection result, and the low-order two's complement data output from the low-order two's complement data output terminal to obtain the low-order operation result data.

2. The serial shift two's complement multiplication and addition circuit with high-order separation according to claim 1, characterized in that, The external signal input port includes a sign bit port and a high-bit calculation start signal port; The first AND gate includes two input terminals, one of which is connected to the sign bit port and the other of which is connected to the high-order calculation start signal port. The first AND gate is used to perform logical operations on the sign bit output from the sign bit port and the high-order calculation start signal, so that the high-order extension signal of the sign bit is obtained when the high-order calculation starts. The first input terminal of the first full adder is connected to the output terminal of the first AND gate, and the second input terminal of the first full adder is connected to the high-order complement data output terminal. The first data selector includes two input terminals. One input terminal is connected to the carry-out terminal of the shift two's complement multiply-adder, and the other input terminal is connected to the carry-out terminal of the first full adder. The first data selector is used to select one of the carry-out data from the output of the shift two's complement multiply-adder and the carry-out data from the output of the first full adder and output it to obtain a first selection result. The input of the first D flip-flop is connected to the output of the first data selector, and the output of the first D flip-flop is connected to the carry input of the first full adder. The first D flip-flop is used to generate a first carry signal based on the first selection result. The first full adder is used to add the high-order extended signal of the sign bit, the high-order complement data output from the high-order complement data output terminal, and the first carry signal to obtain the high-order operation result data.

3. The serial shift two's complement multiplication and addition circuit with high-order separation according to claim 1, characterized in that, The first data selector is specifically used for: When the enable signal of the first data selector is logic 1, the output of the first data selector outputs the carry data of the second full adder. When the enable signal of the first data selector is logic 0, the output of the first data selector outputs the carry data from the first full adder.

4. The serial shift two's complement multiplication and addition circuit with high-order separation according to claim 1, characterized in that, The second data selector is specifically used for: When the enable signal of the second data selector is logic 1, the output of the second data selector outputs the shifted and inverted data. When the enable signal of the second data selector is logic 0, the output of the second data selector outputs the shifted arithmetic data itself.

5. The serial shift two's complement multiplication and addition circuit with high-order separation according to claim 1, characterized in that, The third data selector is specifically used for: When the enable signal of the third data selector is logic 1, the output of the third data selector outputs the carry data of the second full adder; When the enable signal of the third data selector is logic 0, the output of the third data selector outputs the sign bit output by the sign bit port.

6. The serial shift two's complement multiplication and addition circuit with high-order separation according to claim 1, characterized in that, The first data selector further includes a first control terminal, which is connected to the counter control signal output terminal. The first data selector is also used to determine the logic number of the enable terminal signal based on the counter control signal output by the counter control signal output terminal.

7. The serial shift two's complement multiplication and addition circuit with high-order separation according to claim 1, characterized in that, The second data selector also includes a second control terminal, which is connected to the sign bit port. The second data selector is also used to determine the number of logic bits of the enable signal based on the sign bit output by the sign bit port.

8. The serial shift two's complement multiplication and addition circuit with high-order separation according to claim 1, characterized in that, The third data selector also includes a third control terminal, which is connected to the addend availability signal output terminal. The third data selector is also used to determine the logic number of the enable terminal signal based on the addend availability signal output by the addend availability signal output terminal.

9. A pulsating array system, characterized in that, It includes multiple serial shift complement multiply-add operation circuits with high-order separation as described in any one of claims 1-8.

Images