A binarized convolutional neural network computing circuit based on parallel addition trees

Abstract

The application provides a binary convolutional neural network calculation circuit based on a parallel addition tree, and belongs to the field of circuit design. The application designs a special calculation unit based on a parallel addition tree to calculate a binary convolutional neural network. In the calculation unit circuit, 0 is used to represent 1 of weight or data, and 1 is used to represent -1 of weight or data. The calculation unit is composed of a parallel addition tree, a data distributor, a plurality of full adders and registers, wherein the parallel addition tree comprises k XOR gates, a plurality of adders and a two-input selector. In addition, the application designs a same weight accumulation module to convert accumulation of n multi-bit data into accumulation of a plurality of n-bit data, so that each layer of the neural network can realize reuse of the calculation unit through one or more invocations of the above-mentioned addition tree, thereby reducing the complexity of the neural network calculation circuit, saving the circuit cost of the calculation unit, and greatly reducing the power consumption and area of the neural network calculation circuit.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of circuit design, and particularly relates to a binary convolutional neural network calculation circuit based on a parallel addition tree. BACKGROUND

[0002] The speech keyword wake-up (Keyword Spotting) task is often used as a switch for complex artificial intelligence tasks, and its always-on feature requires hardware implementation to have low power consumption and high energy efficiency. The speech keyword wake-up generally consists of two parts: speech feature extraction and speech feature recognition. The former is responsible for converting speech signals into feature data and sending them into the recognition system, and the latter classifies speech features through a neural network to determine whether it is a correct keyword.

[0003] With the development of artificial intelligence, the neural network used for keyword wake-up is also constantly developing and iterating. For example, perceptual linear prediction features (PLP) and deep neural networks (DNN) have been used to complete speech classification tasks. Later, convolutional neural networks (CNN) achieved the best results in the speech keyword wake-up task, and their flexible structure made hardware acceleration more widely used; Convolutional recurrent neural networks (CRNN) have better temporal connectivity than convolutional neural networks, and have better performance in longer keyword classification and context-assisted judgment tasks.

[0004] For hardware circuit design, accuracy, parameter quantity, and computational complexity need to be considered, so the academic and industrial communities have tried and evaluated the hardware cost of various networks. In summary, convolutional neural networks can flexibly change network layers, data bit width, and other parameters, and have a wide receptive field, so they are very suitable for simple speech keyword recognition tasks. In addition, the hardware implementation of neural networks often cannot use full-precision data types for operations, because full-precision will have a lot of precision redundancy in hardware implementation, so the weights of the neural network are often quantized, and the binary weight quantization method is more friendly to hardware, which quantizes the weight data into single-bit values, represented only by 0 and 1. Except for the input data being multi-bit data, the network weights and each layer output are single-bit data.

[0005] For the calculation of a commonly used full-precision neural network, a processing element (PE) is generally composed of a multiply-accumulate (MAC) operation, and the product of the weight and the corresponding input is accumulated with a bias. For a network whose weight and activation value are not binarized, the multiplication in the multiply-accumulate operation is realized by a complex multiplication circuit, but a binarized network can realize the multiplication of a single-bit weight and a single-bit data by using a simple gate circuit, thereby greatly reducing the power consumption and area of the neural network calculation circuit. In a common binarized neural network PE array, the accumulation circuit adopts a serial adder chain mode, which has the advantage of using fewer hardware units and realizing accumulation and addition through multi-cycle calculation, and therefore has the disadvantage of long cycle, high number of accumulator register access, and high flip rate of the accumulated sum.

[0006] For a neural network used for speech keyword recognition, the output timing of the speech feature extraction circuit is fixed, that is, a set of speech features is output every N clock cycles, and therefore, if the neural network cannot complete the calculation within N cycles, the neural network module will have to run at a higher frequency. The current design is to use multiple clock signals so that the neural network module works at a higher frequency, thereby completing the calculation within N cycles. This will make the entire circuit need multiple clocks, and in the chip back-end implementation, multiple clock trees will be generated, and asynchronous first input first output (FIFO) memories are needed to connect between two clock domains, thereby increasing the power consumption of the periphery. SUMMARY

[0007] The present application aims to provide a binarized convolutional neural network calculation circuit based on a parallel addition tree, which designs a dedicated PE calculation unit composed of a parallel addition tree, a plurality of full adders, and a plurality of registers. The parallel addition tree is composed of n (n is a multiple of 3) XOR gates, a plurality of adders, and a two-input selector. At the same time, the present application designs a same weight accumulation module, and the accumulation of multi-bit data can also reuse the above-mentioned parallel addition tree. The convolution calculation and fully connected calculation involved in the entire neural network circuit calculation process are realized by one or more calls to the above-mentioned addition tree, thereby realizing the reuse of the calculation unit and reducing the complexity of the neural network calculation circuit. Compared with the traditional multiply-accumulate unit without optimization, the circuit cost of the calculation unit is saved, and the power consumption and area of the neural network calculation circuit are greatly reduced. The operation of parallel accumulation reduces the number of calculation cycles of the binarized neural network, and in the application of speech keyword wake-up, the calculation of the neural network part needs to be completed within the calculation time of two frames of speech features, and therefore the present application can reduce the calculation frequency required by the neural network circuit.

[0008] To solve the above technical problems, the specific technical solutions of the present application are as follows:

[0009] A binary convolutional neural network calculation circuit based on parallel addition tree, comprising a parallel addition tree and a same weight accumulation module;

[0010] The parallel addition tree is used for completing single-bit data accumulation of the neural network, and an output end of the parallel addition tree is connected to an input end of the same weight accumulation module.

[0011] The same weight accumulation module is used for selecting a bit number of shifting during shifting accumulation and adding the shifting results, and the specific function is to convert accumulation of n m-bit data into accumulation of several n-bit data, wherein n is a multiple of 3, m is a positive integer greater than 2, and n > m.

[0012] Further, the parallel addition tree comprises n XOR gates, a plurality of compressors, an adder and a two-input selector; n weight data and n feature data pass through the n XOR gates to obtain n XOR results, and then are connected to the plurality of compressors; outputs of the compressors are connected to inputs of the adder, and the compressors and the adder are used to obtain accumulation values of the n XOR results; an output end of the adder and an output end of the first compressor are connected to input ends of the two-input selector, and are used to directly output an accumulation result of the three 1-bit data when there are only three 1-bit data.

[0013] Further, the same weight accumulation module comprises a data distributor, a plurality of full adders and a register; an input end of the data distributor is connected to an output end of the parallel addition tree, output ends of the plurality of full adders are connected to an input end of the register, and an output end of the register is connected to an input end of the data distributor; the data distributor has four bit selection signals, which determine a bit number of left shifting of input data; and finally, the data after shifting is added to obtain a final result.

[0014] As a binary convolutional neural network calculation circuit based on a parallel addition tree, compared with a common binary convolutional neural network calculation circuit, the following improvements are made:

[0015] A binary convolutional neural network calculation circuit based on a parallel addition tree, compared with a common binary convolutional neural network calculation circuit, adopts a reusable PE unit based on a parallel addition tree to complete calculation of all network layers. The parallel addition tree is composed of a plurality of 3-2 compressors and a plurality of multi-bit adders, and can directly output a result of three input data passing through the 3-2 compressors through a selector.

[0016] A binary convolutional neural network calculation circuit based on a parallel addition tree, compared with a common binary convolutional neural network calculation circuit, a same weight accumulation module is designed, and n m-bit data accumulation is converted into accumulation of several n-bit data through the addition tree and the shifter, so that the parallel addition tree can be reused for multi-bit data.

[0017] The binarization convolutional neural network computing circuit based on the parallel addition tree has the following advantages:

[0018] (1) Compared with the traditional neural network computing circuit, the computing unit of the application is realized by using parallel addition trees, gate circuits, compressors, etc. Compared with the previous multiply-accumulate computing unit composed of multipliers and adders, the correct calculation result can be obtained only by using the addition tree, and the area is small and the power consumption is low.

[0019] (2) Compared with the traditional neural network computing circuit, the same weight accumulation module designed in the application can convert the accumulation of n m-bit data into the accumulation of several n-bit data, so that the calculation of multi-bit input data and weights can also reuse the parallel addition tree, and the reuse of the computing unit for all layers of the network can reduce unnecessary hardware circuit overhead.

[0020] (3) Compared with the traditional neural network computing circuit, the binarization convolutional neural network computing circuit based on the parallel addition tree can reduce the calculation period of the binarization neural network through high-parallel calculation, thereby reducing the working frequency of the neural network circuit and further reducing the power consumption of the neural network circuit. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a specific application scenario diagram of the application;

[0022] Figure 2 is a soft and hardware data corresponding truth value diagram of the application;

[0023] Figure 3 is a specific neural structure diagram to which the application is applied;

[0024] Figure 4 is a PE computing unit circuit diagram of the application;

[0025] Figure 5 is a same weight accumulation module schematic diagram of the application;

[0026] Figure 6 is a parallel addition tree circuit structure diagram of the application. DETAILED DESCRIPTION

[0027] In order to better understand the purpose, structure and function of the application, the application of a binarization convolutional neural network computing circuit based on a parallel addition tree will be further described in detail below in combination with the drawings.

[0028] A binarization convolutional neural network computing circuit based on a parallel addition tree, the circuit comprising a parallel addition tree and a same weight accumulation module;

[0029] Parallel adder tree, used for completing single-bit data accumulation of neural network, output end of parallel adder tree is connected with input end of same weight value accumulation module. Parallel adder tree contains n XOR gates, several compressors, adder and a two-input selector; n weight data and n feature data are input into n XOR gates to obtain n XOR results, then the n XOR results are input into several compressors; output of the compressors is connected with input of the adder, the compressors and the adder are used to obtain accumulation value of the n XOR results; output of the adder and output of the first compressor are connected with input of the two-input selector, which is used to directly output accumulation result of the three 1-bit data when there are only three 1-bit data.

[0030] Parallel adder tree (as shown in Figure 6 ), which is used for calculating multiplication and accumulation result of feature value and weight value of neural network, each layer of the neural network reuses the same parallel adder tree to reduce overall cost of the circuit. The core of PE calculation unit of the binary neural network is XOR accumulation unit. In the present application, n-bit feature data (Feature) and n-bit weight data (Weight) are first input into XOR gates to obtain n-bit XOR result (Product). The XOR result is accumulated every 3 bits to obtain 2-bit data. Then the 2-bit data are added two by two to obtain 3-bit data. In this way, the 3-bit data are added to obtain 4-bit data, and finally an x-bit output value (x is the smallest positive integer greater than log2n) is obtained. In addition, the parallel adder tree can directly output the result of 3-2 compressor of the three inputs through the selector to complete the calculation of the convolution per channel.

[0031] Same weight value accumulation module, used for selecting number of bits of shifting when shifting and accumulating, and adding the shifted result, and the specific function is to convert accumulation of n m-bit data into accumulation of several n-bit data, wherein n is a multiple of 3, m is a positive integer greater than 2, and n>m. The same weight value accumulation module contains a data distributor, several full adders and a register, the output end of the parallel adder tree is connected with the input end of the data distributor, the output end of the data distributor is connected with the input end of the several full adders, and the output end of each full adder is connected with the input end of the register; the data distributor has four bits of selection signal, which determines the number of bits of left shifting of the input data; and finally the data after shifting is added to obtain the final result.

[0032] Same weight value accumulation module (as shown in Figure 5 ), which is used for converting accumulation of n m-bit data into summation of several n-bit accumulation values.

[0033] First, the bits with the same weight for these n values ​​are summed, and from the least significant bit to the most significant bit, we can obtain ACC0, ACC1, ..., ACC m-1 These are the accumulated values ​​corresponding to the m weights. The bit width of the n 1-bit accumulated values ​​is x bits (x is the smallest positive integer greater than log₂n). The purpose of the same weight accumulation module is to use ACC0, ACC1, ..., ACC m-1 Obtain the correct calculation result for x bits.

[0034] Then multiply the m accumulated values ​​by their corresponding weights, i.e., ACC0 × 2 0 ACC1×2 1 ACC2×2 2 , ..., ACC m-1 ×2 m-1 This is equivalent in hardware to shifting each accumulated value accordingly. Considering the padding of the sign bit in signed numbers, the sign bit needs to be processed separately. As shown in the following formula, ANS is the result of calculating these n m-bit signed numbers, where... Let A represent the i-th signed number. i This represents the unsigned value corresponding to the two's complement. This represents the sign bit data of the i-th signed number.

[0035]

[0036] For m-bit data, the conversion relationship between signed and unsigned numbers is as follows:

[0037]

[0038] Therefore, the final ANS can be seen as the accumulated value of the unsigned number minus the compensation value, as shown in the following formula, where << m represents a left shift of m bits:

[0039]

[0040] When the input data of the first convolutional layer is multi-bit, the weighted accumulation module first transforms the accumulation of n m-bit data into the accumulation of several n-bit data through multi-cycle parallel accumulation and shifting. Then the obtained n-bit data can reuse the parallel addition tree, ultimately achieving the effect that the same addition tree can be reused for each convolutional operation.

[0041] The present invention discloses a binary convolutional neural network computation circuit based on parallel addition trees, designed for speech keyword recognition scenarios. Each frame of speech input in chronological order corresponds to a set of speech feature data. Several sets of speech feature data are processed by the neural network circuit to obtain the output result (e.g., ...). Figure 1The circuit is used for the structure of the convolutional neural network and its variants, such as the deep separable convolutional neural network (DSCNN, including channel-wise convolution and point-wise convolution), the time convolutional neural network (TCN), and the specific network is introduced below.

[0042] In the PE unit proposed in the design, 0 is used on the hardware circuit to represent the actual weight or data 1, and 1 represents the actual weight or data -1. The truth table of the calculation is as shown in the figure Figure 2 The specific neural network structure applied in the design is as shown in the figure Figure 3 The specific neural network structure applied in the design is as shown in the figure Figure Four The connection mode is as follows: the output result of the first layer convolution (CNN) is used as the input feature value of the first group of DSCNN, wherein the output of the channel-wise convolution is the input of the point-wise convolution; the output result of the first group of DSCNN is used as the input feature value of the second group of DSCNN; the output result of the second group of DSCNN is used as the input feature value of the third group of DSCNN; the output result of the third group of DSCNN is used as the input data of the fully connected layer (FC). The parallel addition tree corresponding to the specific example case is composed of 48 XOR gates, 16 3-2 compressors, 8 2-bit adders, 4 3-bit adders, 2 4-bit adders, one 5-bit adder, and one two-input selector, as shown in the figure Figure 6 The 48 weight data and 48 feature data pass through 48 XOR gates to obtain 48 XOR results, and then are connected to 16 compressors; the outputs of the 16 compressors are connected to the inputs of 8 three-bit adders to obtain 8 output values connected to the inputs of 4 four-bit adders, and the outputs thereof are connected to the input terminals of 2 five-bit adders, and finally are connected to the input terminal of one six-bit adder; the output terminal of the six-bit adder and the output terminal of the first compressor are connected to the input terminals of the two-input selector, which is used to directly output the accumulation result of the three one-bit data when the input is only three one-bit data.

[0043] (1) First layer convolution

[0044] The first layer convolution operation of the neural network has 48 8-bit data (n=48, m=8) as the input, and 48 1-bit data as the weight. After multiplication of the input and the weight, 48 8-bit data are obtained, so the calculation of the first layer convolution is the accumulation of 48 8-bit data. In order to reuse the addition tree, the accumulation of 48 8-bit data needs to be converted to the accumulation of 48-bit data through the same weight accumulation module. The schematic diagram of the same weight accumulation module is as shown in the figure Figure FiveAs shown. First, before accumulation, the compensated weights are assigned to the accumulation register. Each time a frame of speech feature values ​​is input, the bits are sequentially input from least significant to most significant. Figure Six The addition tree shown accumulates the sums, resulting in 8-bit accumulated values ​​ACC0, ACC1, ..., ACC7. Then, based on the weights, the addition tree outputs are assigned to different positions within the accumulated value. The specific circuit structure is as follows... Figure 4 As shown, the 48-bit weight and activation values ​​are processed by a parallel addition tree to obtain a 6-bit cumulative result. The cumulative result is then processed by a data distributor to determine the specific number of bits to shift. The 6-bit cumulative values ​​(ACC0, ACC1, ..., ACC7) are shifted and added together to finally obtain a 14-bit output value.

[0045] (2) Channel-wise convolution

[0046] like Figure 3 As shown, for channel-wise convolution, the weights on each channel are three 1-bit data points, so the corresponding operation is equivalent to the sum of three 1-bit numbers. Since channel-wise convolution only requires calculating the sum of three 1-bit numbers, the last layer of the addition tree has a two-input selector that determines whether to output the sum of 48 1-bit input data points or the sum of 3 inputs obtained from the first 3-2 compressor. Figure 4 The diagram shows the bottom layer of a parallel adder tree circuit. After passing through the adder tree, a 6-bit accumulated value is obtained. Figure 4 In the PE calculation unit shown, the output of the addition tree is output as the calculation result of the channel-by-channel convolutional layer.

[0047] (3) Pointwise convolution

[0048] like Figure 3 As shown, for pointwise convolution, the weight size is 48 groups of 1-bit data, with each group containing 48 1-bit numbers. These 48 data points are multiplied and accumulated with the input of the pointwise convolution. Therefore, this step is equivalent to accumulating 48 1-bit numbers. Figure Six As shown, 48 1-bit data points are fed into the addition tree, resulting in a 6-bit cumulative sum, which is used as the output of the pointwise convolutional layer.

[0049] (4) Fully connected

[0050] The weights of the fully connected layer are 288 1-bit data. Multiplying and accumulating them with the output of the last pointwise convolution is equivalent to accumulating 288 1-bit numbers. Therefore, it is necessary to perform 6 addition tree calculations and accumulate the results of these 6 calculations to obtain the output of the fully connected layer.

[0051] In summary, the PE computing unit can multiplex the four computing modes in the network at a minimum cost, and the parallel addition tree can effectively reduce the computing period. After using the PE array with the parallel addition tree, if only the computation is considered, the entire neural network only needs 21 periods to complete a computation, so the parallel addition tree can greatly reduce the running time, thereby enabling the neural network to carry the data throughput of the feature extraction module at the same clock.

[0052] The application is simulated at 28nm, 0.5V, TT corner, 25℃ environment. Since the computing unit based on the parallel addition tree can reduce the computing period, the neural network part can work at 8kHz and ensure that the computation of the neural network part is completed before the next frame of speech feature data arrives. Finally, the power consumption of the neural network circuit using the computing unit described in the application is 305nW.

[0053] Compared with the circuit using the traditional computing unit (multiply-accumulate PE unit) and having the same neural network structure (published in ESSCIRC 2021, AAD-KWS: an always-on keyword spotting chip with a zero-cost, deep-coupled acoustic activity detector from a 170nW MFCC feature extractor in 28nm CMOS), if the computing unit is ensured to complete the computation of the neural network part before two frames of speech feature data arrive, the frequency needs to be increased to 200kHz, and the power consumption is 630nW.

[0054] In contrast, the binary convolutional neural network computing circuit based on the parallel addition tree described in the application can reduce the power consumption of the neural network circuit by 51.5%, and the comparison table is as follows:

[0055]

[0056] Table 1 Comparison table of neural network circuit performance with different computing units

[0057] It can be understood that the present application is described through some embodiments, and those skilled in the art know that various changes or equivalent replacements can be made to these features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, these features and embodiments can be modified to adapt to specific conditions and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope of the present application.

Claims

1. A binary convolutional neural network computation circuit based on a parallel addition tree, characterized in that, The circuit includes a parallel addition tree and a weighted accumulation module; Parallel addition trees are used to perform single-bit data accumulation in neural networks. The output of the parallel addition tree is connected to the input of the same weight accumulation module. The equal-weighted accumulation module is used to select the number of bits to shift during shift accumulation and add the shift results. Specifically, it converts the accumulation of n m-bit data into the accumulation of several n-bit data, where n is a multiple of 3, m is a positive integer greater than 2, and n>m.

2. The binary convolutional neural network computation circuit based on parallel addition trees according to claim 1, characterized in that, The parallel addition tree contains n XOR gates, several compressors, adders, and a two-input selector. n weight data and n feature data are passed through the n XOR gates to obtain n XOR results, which are then connected to the compressors. The outputs of the compressors are connected to the inputs of the adders. The compressors and adders are used to obtain the accumulated value of the n XOR results. The outputs of the adders and the first compressor are connected to the inputs of the two-input selector, used to directly output the accumulated result of these three 1-bit data when the input is only three 1-bit data.

3. The binary convolutional neural network computation circuit based on parallel addition trees according to claim 1, characterized in that, The equal-weight accumulation module includes a data distributor, several full adders, and a register. The input of the data distributor is connected to the output of the parallel addition tree, and the output of the data distributor is connected to the input of several full adders. The output of each full adder is connected to the input of a register. The data distributor has a four-bit strobe signal to determine the number of bits to shift the input data to the left. Finally, the shifted data are added together to obtain the final result.

Images