Deep neural network mixing precision hardware acceleration circuit based on probability calculation
By introducing error compensation probability multiplier and shift circuit in hybrid accuracy operation, the problems of large area, high delay and high power consumption of the hybrid accuracy operation processing unit are solved, and more efficient deep neural network model acceleration is achieved.
Patent Information
- Application Number
- CN202411130749.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-16
AI Technical Summary
The existing hybrid precision calculation processing units have large area, high delay, high power consumption, and have problems such as multiple error and symbol error in probability calculation.
The error compensation probability multiplier is used as the basic operation unit, and a 4-bit hybrid precision processing unit is formed by combining the shift circuit and the adder. The operation is carried out through the fine particle size space parallel probability multiplier and the error compensation mechanism to realize the probability operation of the bitstream sequence. And selective output of symbol errors.
It saves 32.9% area overhead, reduces delay by 18.8%, and reduces power consumption by 59.2%, improves the efficiency of hybrid accuracy computing, reduces errors, and achieves acceleration of deep neural network models.
Smart Images

Figure HDA0004997741600000011 
Figure HDA0004997741600000012 
Figure HDA0004997741600000013
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of integrated circuits and artificial intelligence, and in particular to a mixed-precision hardware acceleration circuit for a deep neural network based on probability calculation. Background Art
[0002] With the rapid development of neural network technology in various fields, its computing requirements are increasing. Various computing architectures are currently used to accelerate the calculation of neural networks. In order to ensure computing power, the traditional computing architecture has a high hardware complexity and a large functional overhead.
[0003] There is a probability calculation system that can reduce the computational cost of neural networks. It uses a traditional probability multiplier as the basic computing unit. Figure 1 As shown, it is capable of probabilistic calculations. It is actually composed of three parts: forward conversion, probability calculation module and backward conversion. The forward conversion module is mainly composed of a random number generator, which is used to generate a random bit stream and encode the number into a sequence of "1" and "0". The probability calculation module performs basic bit operations through a simple logic circuit. The hardware implementation cost of the probability calculation system is low and the error resistance is strong. It is suitable for implementing complex operations or iterative algorithms. However, the dense multiplication and accumulation iterative operations in the neural network hinder its hardware implementation, making the probability calculation system suitable for performing operations on the neural network and reducing the operation cost of the neural network. However, it simplifies the size of the basic operation unit, but sacrifices the calculation accuracy. The randomness of the bit stream also reduces the degree of overfitting problems, so the application of the probability calculation system in accelerating the neural network operation architecture is limited.
[0004] In the existing research direction of computing architecture, low-bit quantization is an important part of transformer network compression. By converting floating-point weights and activations to low-bit fixed-point numbers, the size of parameters can be significantly reduced. Existing solutions have proposed a variety of post-training methods to quantize activations and weights to INT8 / INT6 format. However, applying uniform precision to all network layers ignores the sensitivity of different parts of the underlying neural network system to different local performances of quantization errors.
[0005] Another method is precision scalable quantization, which takes advantage of the different degrees of robustness and fault tolerance of neural network layers. Based on this quantization method, a variety of precision scalable structures can be designed to quantize deep neural network models on mobile terminals. In order to adapt low-bit quantization on hardware, the authors of the document [1] proposed a bit-flexible DNN accelerator called BitFusion, which minimizes bit-granularity calculations and communications without sacrificing accuracy. Subsequent researchers developed the SumTogether mode based on the idea of bit fusion and set 2 / 4 / 8 / 16-bit precision to implement DNN processing. They also proposed a precision scalable 2 / 4 / 8 / bit vector system accelerator based on bit splitting and combining, namely BSC.
[0006] [1]H.Sharma et al., "Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network," 2018ACM / IEEE 45th AnnualInternational Symposium onComputer Architecture(ISCA), Los Angeles, CA, USA, 2018, pp.764-775, doi:10.1109 / ISCA.2018.00069.
[0007] In traditional probability calculation, there are problems such as long conversion time and high encoder overhead. For binary input data, it must first be converted into a number domain. The length of the traditional sequence after conversion is 2. n As the number of bits increases, there is room for optimization of this part of the circuit in terms of time and hardware complexity.
[0008] In the existing mixed-precision architecture design, each sub-unit expands 1 sign bit, that is, a 3-bit multiplier is used in each sub-unit. As the bit width increases, the number of 3-bit multipliers required also increases exponentially, which results in a larger mixed-precision operation processing area, high latency, and high power consumption. Summary of the invention
[0009] The purpose of the present invention is to provide a mixed-precision hardware acceleration circuit for a deep neural network based on probability calculation, to improve the shortcomings of the existing mixed-precision operation processing unit, such as large area, high delay, and high power consumption, and to improve the shortcomings of the existing fine-grained spatial parallel probability multiplier, such as the existence of multiple errors and sign errors in the probability calculation, to accelerate the probability calculation of the deep neural network model.
[0010] The present invention provides a mixed precision hardware acceleration circuit for a deep neural network based on probability calculation, characterized in that an error compensation probability multiplier is used as a basic operation unit, and four of the error compensation probability multipliers are combined with a shift circuit and an adder to form a 4-bit mixed precision processing unit.
[0011] Preferably, the error compensation probability multiplier consists of two parts: a fine-grained spatial parallel probability multiplier and an error compensation mechanism.
[0012] The fine-grained spatial parallel probability multiplier includes a forward conversion module, a calculation module, and a backward conversion module. The forward conversion module uses weight extension coding and uniform coding to convert a binary bit stream sequence into an equal-weighted bit stream sequence. The calculation module performs probability calculations, and the backward conversion module converts the equal-weighted bit stream sequence back into a binary sequence. Let two 2-bit multipliers be denoted as A and W, A uses weight extension coding, and W uses uniform coding. The coded sequence is expressed by the formula:
[0013] A=(a1a0) bin =(a1a1a0) sto
[0014] W=(w1w0) bin =(w1w0w1) sto
[0015] Where A and W are two 2-bit multipliers, a0 and a1 are the 2 bits of multiplier A, w0 and w1 are the 2 bits of multiplier W, (·) bin Represents the value of a binary field, (·) sto Represents the numerical value of the probability domain, (a1a0) bin is the two's complement representation of the multiplier A, (a1a1a0) sto is the probability sequence representation after weight expansion coding, (w1w0) bin is the two's complement representation of the multiplier W, (w1w0w1) sto It is the probability sequence representation after uniform coding.
[0016] The probability multiplication result of A and W is expressed as:
[0017] A·W=a1w1+a1w0+a0w1
[0018] The error compensation mechanism utilizes a shift module to perform multiple error compensation on the operation result of the fine-grained spatial parallel probability multiplier, and designs a gating mechanism to separate the output of signed numbers and unsigned numbers to compensate for the error in the operation sign.
[0019] Preferably, in the error compensation probability multiplier, if Figure 2 As shown, the port connection relationship between the fine-grained spatial parallel probability multiplier and the error compensation mechanism is that the input end of the error compensation probability multiplier is the bit positions a0, a1, w0, w1 split from two 2-bit data A and W, and a circuit input selection signal C in , where the two bits of A are a0 and a1, and the two bits of W are w0 and w1. After the fine-grained spatial parallel probability multiplier, the outputs a1w1, a1w0, and a0w1 are obtained. These three outputs are connected to the error compensation mechanism. After the output of a1w1, the shift 2 circuit in the error compensation mechanism is connected. After the output of a0w1 and a1w0, the shift 1 circuit in the error compensation mechanism is connected. The shifted output 2 2 a1w1,2 1 a1w0,2 1 a0w1, the three shifted outputs are connected to the adder and subtractor, and the output unsigned and signed probability operation results (A W) s_sto and (A.W) u_sto , the probability calculation result and C in In the common input gate, (A·W) s_sto or (A.W) u_sto Output.
[0020] Preferably, the error compensation mechanism uses a shift module to perform multiple error compensation on the operation result of the fine-grained spatial parallel probability multiplier. In a traditional fine-grained spatial parallel probability multiplier, a backward conversion module converts a bit stream sequence of equal weights into a binary sequence. The traditional 2-bit unsigned multiplication operation is expressed by the formula:
[0021] (A.W) u_bin =2 2 a1w1+2 1 a1w0+2 1 a0w1+a0w0
[0022] in,(·) u_bin Represents the result of unsigned binary operation. The traditional 2-bit signed multiplication operation is expressed as:
[0023] (A.W) s_bin =2 2 a1w1-2 1 a1w0-2 1 a0w1+a0w0
[0024] in,(·) s_bin Represents the result of a signed binary operation.
[0025] Theoretically, the probability multiplication result of A and W is expressed as:
[0026] A·W=a1w1+a1w0+a0w1
[0027] It can be seen that when error compensation is not performed, the coefficient of probability multiplication is in a multiple relationship with the theoretical coefficient. Directly using the probability calculation result will result in a multiple error. At this time, the error compensation mechanism is designed to ensure the characteristics of the original data.
[0028] In digital circuits, shifting a bit to the left by 1 bit can realize a binary multiplication operation by 2. The shift module is connected to a 2-bit shift circuit after the output of a1w1, and a 1-bit shift circuit is connected to the output of a0w1 and a1w0. The output results are sent to the adder and subtractor respectively to obtain the probability operation results of unsigned numbers and signed numbers. The probability multiplication result after compensation is expressed by the formula:
[0029] (A.W) s_sto =2 2 a1w1-2 1 a1w0-2 1 a0w1
[0030] (A.W) u_sto =2 2 a1w1+2 1 a1w0+2 1 a0w1
[0031] in,(·) s_sto Represents the probability operation result of a signed number, (·) u_sto The probability operation result representing an unsigned number. The corrected probability multiplication will have an error only when a0=1, w0=1, and the maximum error is 1.
[0032] Preferably, the error compensation mechanism is designed to compensate for the error in the operation sign of the traditional fine-grained spatial parallel probability multiplier. The operation result of the traditional fine-grained spatial parallel probability multiplier may have the problem of positive and negative operation errors. A gating mechanism is designed in the error compensation probability multiplier to selectively output the signed number and the unsigned number, thereby compensating for the error in the operation sign. in , and the probability multiplication result after compensation by the shift module (A·W) s_sto and (A.W) u_sto An input selector is used to select whether to output a signed number or an unsigned number, and the selected number is used as the output of the error compensation probability multiplier, and then the selected number is input into the overall mixed precision computing unit for calculation.
[0033] Preferably, four error compensation probability multipliers combined with a shift circuit and an adder form a 4-bit mixed precision processing unit, which means splitting two 4-bit signed numbers into four 2-bit numbers, which can be expressed as:
[0034] M=(M1M0) bin =2 2 M1+M0
[0035] N=(N1N0) bin =2 2 N1+N0
[0036] Among them, M and N are two input 4-bit signed numbers, M0 and M1 are two 2-bit numbers split by M, N0 and N1 are two 2-bit numbers split by N, (·) bin Represents the value of the binary field, (M1M0) bin Represents the two's complement of the multiplier M, (N1N0) bin Represents the binary complement of the multiplier N, and each coefficient is realized by a shift circuit.
[0037] like Figure 3 As shown, the four 2-bit numbers M1, M0, N1, and N0 are respectively input into the four error compensation probability multipliers, and the output results of the four error compensation probability multipliers are input into the shift circuit, and the adder is used to sum to form a 4-bit mixed precision processing unit. The output of the multiplication of M and N is recorded as K. The 4-bit mixed precision processing unit is expressed by the formula:
[0038] K=M·N=2 4 (M1 N1) s_sto +2 2 (M1 N0) s_sto +2 2 (M0·N1) s_sto +(M0·N0) u_sto
[0039] Where K is the product of M and N, (M1·N1) s_sto 、(M1·N0) s_sto 、(M0·N1) s_sto 、(M0·N0) u_sto They are all realized by the error compensation probability multipliers, which are the outputs of the four error compensation probability multipliers, and each coefficient is realized by a shift circuit.
[0040] Preferably, four of the 4-bit mixed precision processing units are combined with a shift circuit and an adder to form an 8-bit mixed precision processing unit. The two 8-bit signed numbers are recorded as P and Q respectively. The 8-bit number P is decomposed into two 4-bit numbers P1 and P0, and the 8-bit number Q is decomposed into two 4-bit numbers Q1 and Q0, which can be expressed as follows:
[0041] P=(P1P0) bin
[0042] Q=(Q1Q0) bin
[0043] Among them, P and Q are two 8-bit signed multipliers, P1 and P0 are two 4-bit numbers decomposed from the multiplier P, and Q1 and Q0 are two 4-bit numbers decomposed from the multiplier Q, (P1P0) bin is the two's complement of the multiplier P, (Q1Q0) bin is the two's complement of the multiplier Q,
[0044] Among them, P1 can be further decomposed into two 2-bit numbers P 11 , P 10 , P0 can be further decomposed into two 2-bit numbers P 01 , P 00 , Q1 can be further decomposed into two 2-bit numbers Q 11 , Q 10 , Q0 can be further decomposed into two 2-bit numbers Q 01 , Q 00 , expressed as:
[0045] P1=(P 11 P 10 ) bin =2 2 P 11 +P 10
[0046] P0=(P 01 P 00 ) bin =2 2 P 01 +P 00
[0047] Q1=(Q 11 Q 10 ) bin =2 2 Q 11 +Q 10
[0048] Q0=(Q 01 Q 00 ) bin =22 Q 01 +Q 00
[0049] Among them, (P 11 P 10 ) bin is the two's complement of P1, (P 01 P 00 ) bin is the two's complement of P0, (Q 11 Q 10 ) bin is the two's complement of Q1, (Q 01 Q 00 ) bin It is the binary complement of Q0, and each coefficient is realized by the shift circuit.
[0050] like Figure 4 As shown, the split 2-bit data P 01 , P 00 , Q 01 , Q 00 The input is sent to the four 4-bit mixed precision processing units, and the outputs are P1Q1, P1Q0, P0Q1, and P0Q0, respectively, which are expressed as follows:
[0051] P1Q1=2 4 (P 11 Q 11 ) s_sto +2 2 (P 11 Q 10 ) s_sto +2 2 (P 10 Q 11 ) s_sto +2 0 (P 10 Q 10 ) u_sto
[0052] P1Q0=2 4 (P 11 Q 01 ) s_sto +2 2 (P 11 Q 00 ) s_sto +2 2 (P 10 Q 01 ) u_sto +2 0 (P 10 Q 00 ) u_sto
[0053] P0Q1=2 4 (P 01 Q 11 ) s_sto +2 2 (P 01 Q 10 ) u_sto +2 2 (P 00 Q 11 ) s_sto +2 0 (P 00 Q 10 ) u_sto
[0054] P0Q0=2 4 (P 01 Q 01 ) u_sto +2 2 (P 01 Q 00 ) u_sto +2 2 (P 00 Q 01 ) u_sto +2 0 (P 00 Q 00 ) u_sto
[0055] Among them, P1Q1, P1Q0, P0Q1, P0Q0 are the outputs of the four 4-bit mixed precision processing units, (·) s_sto Represents the probability operation result of a signed number, (·) u_sto It represents the probability operation result of an unsigned number. The items on the right side of the equal sign are the outputs of the error compensation probability multiplier in the 4-bit precision mixing unit after passing through the shift circuit. The coefficients of each item are realized by the shift circuit.
[0056] The calculation results P1Q1, P1Q0, P0Q1, and P0Q0 of the four 4-bit mixed precision processing units are input into the adder through the shift circuit, and the calculation results of the 8-bit numbers P and Q are output, which are expressed as follows:
[0057] P·Q=2 8 P1Q1+2 4 P1Q0+2 4 P0Q1+P0Q0
[0058] Among them, each coefficient is realized by a shift circuit.
[0059] Compared with the prior art, the present invention has the following beneficial effects:
[0060] 1. The present invention provides a mixed-precision hardware acceleration circuit for a deep neural network based on probability calculation, which uses an error compensation probability multiplier as a basic operation unit, makes the maximum error of the probability operation 1, and selectively outputs signed numbers and unsigned numbers, thereby improving the deficiencies of multiple errors and sign errors existing in existing fine-grained spatial parallel probability multipliers;
[0061] 2. The present invention provides a mixed-precision hardware acceleration circuit for a deep neural network based on probability calculation, which takes an error compensation probability multiplier as a basic operation unit. Four of the error compensation probability multipliers are combined with a shift circuit and an adder to form a 4-bit mixed-precision processing unit. Compared with the existing mixed-precision operation processing unit, it saves 32.9% of the area overhead, reduces the delay by 18.8%, and reduces the power consumption by 59.2%, which can improve the shortcomings of the existing mixed-precision operation processing unit, such as large area, high delay, and high power consumption.
[0062] 3. The present invention provides a mixed-precision hardware acceleration circuit for a deep neural network based on probability calculation, which takes an error compensation probability multiplier as a basic operation unit. Four of the error compensation probability multipliers are combined with a shift circuit and an adder to form a 4-bit mixed-precision processing unit, which performs probability operations on bit stream sequences and simplifies the structure of existing mixed-precision operation processing units, thereby achieving the beneficial effect of accelerating the probability operations of deep neural network models. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] Figure 1 This is the schematic diagram of the traditional probability multiplier.
[0064] Figure 2 Schematic diagram of the error compensation probability multiplier.
[0065] Figure 3 This is the structural diagram of the 4-bit mixed precision processing unit.
[0066] Figure 4 This is a structural diagram of the mixed-precision acceleration circuit for deep neural networks based on probability calculation. DETAILED DESCRIPTION
[0067] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.
[0068] Example 1
[0069] This embodiment proposes a mixed precision hardware acceleration circuit for a deep neural network based on probability calculation, including 16 of the error compensation probability multipliers, 4 of the error compensation probability multipliers combined with a shift circuit to form a 4-bit mixed precision processing unit, and 4 of the 4-bit mixed precision processing units combined with a shift circuit to form an 8-bit mixed precision processing unit.
[0070] Perform 2-bit precision calculations, input the binary bit stream into the mixed-precision hardware acceleration circuit of the deep neural network based on probability calculation, firstly, the forward conversion module of the fine-grained spatial parallel probability multiplier converts the binary bit stream sequence into a bit stream sequence with equal weights, and the two 2-bit multipliers to be multiplied are respectively denoted as A and W, A is encoded using weight extension, and the encoded sequence is expressed as:
[0071] A=(a1a0) bin =(a1a1a0) sto
[0072] Among them, a1 and a0 are the two bits of the multiplier A, (·) bin Represents the value of a binary field, (·) sto Represents the numerical value of the probability domain, (a1a0) bin is the two's complement representation of the multiplier A, (a1a1a0) sto It is the probability sequence representation after weight expansion encoding.
[0073] W uses uniform coding, and the encoded sequence is expressed as follows:
[0074] W=(w1w0) bin =(w1w0w1) sto
[0075] Among them, w1, w0 are the 2 bits of the multiplier W, (w1w0) bin is the two's complement representation of the multiplier W, (w1w0w1) sto It is the probability sequence representation after uniform coding. A and W pass through the operation module, and the result of probability multiplication is expressed as follows:
[0076] A·W=a1w1+a1w0+a0w1
[0077] The equal-weight bit stream sequence obtained by the operation is sent to the backward conversion module, converted back to a binary bit stream sequence, and then the bit stream sequence is shifted through the error compensation mechanism, and the output results are sent to the adder and subtractor respectively to obtain the operation results of unsigned numbers and signed numbers. The probability multiplication result after compensation is expressed by the formula:
[0078] (A.W) s_sto =22 a1w1-2 1 a1w0-2 1 a0w1
[0079] (A.W) u_sto =2 2 a1w1+2 1 a1w0+2 1 a0w1
[0080] in,(·) s_sto Represents the result of a signed number probability operation, (·) u_sto Represents the result of unsigned probability operation.
[0081] Perform 4-bit precision calculations. Let M and N be two 4-bit signed inputs. Then M is split into two 2-bit numbers M1 and M0, and N is split into two 2-bit numbers N1 and N0. The formula is:
[0082] M=(M1M0) bin =2 2 M1+M0
[0083] N=(N1N0) bin =2 2 N1+N0
[0084] The four 2-bit numbers are respectively input into the four error compensation probability multipliers, and the output calculation results are input into the shift circuit, and summed by the adder to form a 4-bit mixed precision rough reasonable unit. The result of multiplying M and N can be recorded as K, and the probability calculation result of the 4-bit mixed precision processing unit is expressed by the formula:
[0085] K=M·N=2 4 (M1 N1) s_sto +2 2 (M1 N0) s_sto +2 2 (M0·N1) s_sto +(M0·N0) u_sto
[0086] Among them, (M1 N1) s_sto 、(M1·N0) s_sto 、(M0·N1) s_sto 、(M0·N0) u_sto All are realized by the error compensation probability multiplier, and each coefficient is realized by a shift circuit.
[0087] Perform 8-bit precision calculations. The two 8-bit signed numbers are input as P and Q. The 8-bit input P is decomposed into two 4-bit numbers P1 and P0, and the 8-bit input Q is decomposed into two 4-bit numbers Q1 and Q0, which can be expressed as follows:
[0088] P=(P1P0) bin
[0089] Q=(Q1Q0) bin
[0090] Among them, P1 is further decomposed into two 2-bit numbers P 11 , P 10 , P0 can be further decomposed into two 2-bit numbers P 01 , P 00 , and Q1 can be further decomposed into two 2-bit numbers Q 11 , Q 10 , Q0 can be further decomposed into two 2-bit numbers Q 01 , Q 00 , expressed as:
[0091] P1=(P 11 P 10 ) bin =2 2 P 11 +P 10
[0092] P0=(P 01 P 00 ) bin =2 2 P 01 +P 00
[0093] Q1=(Q 11 Q 10 ) bin =2 2 Q 11 +Q 10
[0094] Q0=(Q 01 Q 00 ) bin =2 2 Q 01 +Q 00
[0095] like Figure 3 As shown, the 2-bit data P after splitting 01 , P 00 , Q 01 , Q 00For example, the output results are input into four error compensation probability multipliers respectively, and then passed through a shift circuit and input into an adder to form a 4-bit mixed precision processing unit. Figure 3 The circuits in the four dotted boxes are the four 4-bit mixed precision processing units, which are expressed by the formula:
[0096] P1Q1=2 4 (P 11 Q 11 ) s_sto +2 2 (P 11 Q 10 ) s_sto +2 2 (P 10 Q 11 ) s_sto +2 0 (P 10 Q 10 ) u_sto
[0097] P1Q0=2 4 (P 11 Q 01 ) s_sto +2 2 (P 11 Q 00 ) s_sto +2 2 (P 10 Q 01 ) u_sto +2 0 (P 10 Q 00 ) u_sto
[0098] P0Q1=2 4 (P 01 Q 11 ) s_sto +2 2 (P 01 Q 10 ) u_sto +2 2 (P 00 Q 11 ) s_sto +2 0 (P 00 Q 10 ) u_sto
[0099] P0Q0=2 4 (P 01 Q 01 ) u_sto +2 2(P 01 Q 00 ) u_sto +2 2 (P 00 Q 01 ) u_sto +2 0 (P 00 Q 00 ) u_sto
[0100] The calculation results of the four 4-bit mixed precision processing units are input into the adder through the shift circuit to form an 8-bit mixed precision processing unit. The probability calculation result is expressed by the formula:
[0101] P·Q=2 8 P1Q1+2 4 P1Q0+2 4 P0Q1+P0Q0
[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A mixed-precision hardware acceleration circuit for a deep neural network based on probability calculation, characterized in that: The error compensation probability multiplier is used as a basic operation unit, and four of the error compensation probability multipliers are combined with a shift circuit and an adder to form a 4-bit mixed precision processing unit.
2. According to claim 1, a mixed precision hardware acceleration circuit for a deep neural network based on probability calculation is characterized in that: The error compensation probability multiplier consists of two parts: a fine-grained spatial parallel probability multiplier and an error compensation mechanism. The fine-grained spatial parallel probability multiplier includes a forward conversion module, a calculation module, and a backward conversion module, and uses weight extension coding and uniform coding to convert a binary bit stream sequence into a bit stream sequence with equal weights. The error compensation mechanism utilizes a shift module to perform multiple error compensation on the operation result of the fine-grained spatial parallel probability multiplier, and designs a selection mechanism to selectively output signed numbers and unsigned numbers to compensate for the error in the operation sign.
3. According to claim 2, a mixed precision hardware acceleration circuit for a deep neural network based on probability calculation is characterized in that: The port connection relationship between the fine-grained spatial parallel probability multiplier and the error compensation mechanism, The input end of the error compensation probability multiplier is the bits a0, a1, w0, w1 split from two 2-bit data A and W, and a circuit input selection signal C in , where the two bits of A are a0 and a1, and the two bits of W are w0 and w1. After the fine-grained spatial parallel probability multiplier, the outputs a1w1, a1w0, and a0w1 are obtained. These three outputs are connected to the error compensation mechanism. After the output of a1w1, the shift 2 circuit in the error compensation mechanism is connected. After the output of a0w1 and a1w0, the shift 1 circuit in the error compensation mechanism is connected. The shifted output 2 2 a1w1,2 1 a1w0,2 1 a0w1, the three shifted outputs are connected to the adder and subtractor, and the output unsigned and signed probability operation results (A W) s_sto and (A.W) u_sto , the probability calculation result and C in In the common input gate, (A·W) s_sto or (A.W) u_sto Output.
4. According to claim 2, a mixed precision hardware acceleration circuit for a deep neural network based on probability calculation is characterized in that: The error compensation mechanism uses a shift module to perform multiple error compensation on the operation result of the fine-grained spatial parallel probability multiplier. A 2-bit shift circuit is connected after the output of a1w1, and a 1-bit shift circuit is connected after the output of a0w1 and a1w0. The output results are sent to the adder and subtractor respectively to obtain the probability operation results of unsigned numbers and signed numbers. The probability multiplication result after compensation by the shift module is expressed by the formula: (A·W) s_sto =2 2 a1w1-2 1 a1w0-2 1 a0w1 (A·W) u_sto =2 2 a1w1+2 1 a1w0+2 1 a0w1 Where A and W are two 2-bit multipliers, a0 and a1 are the 2 bits of multiplier A, w0 and w1 are the 2 bits of multiplier W, (·) s_sto Represents the probability operation result of a signed number, (·) u_sto Represents the probability operation result of an unsigned number, 2 2 a1w1,2 1 a1w0,2 1 a0w1 is the output after passing through the shift module, and each coefficient is realized by the shift circuit.
5. The mixed precision hardware acceleration circuit of a deep neural network based on probability calculation according to claim 2, characterized in that: The error compensation mechanism is to design a gating mechanism in the error compensation probability multiplier to selectively output the signed number and the unsigned number, to compensate for the error in the operation sign, and to input the selection signal C of the circuit. in , and the probability multiplication result after compensation by the shift module (A·W) s_sto and (A.W) u_sto An input selector is used to select whether to output a signed number or an unsigned number, and the selected number is used as the output of the error compensation probability multiplier.
6. The mixed precision hardware acceleration circuit for a deep neural network based on probability calculation according to claim 1, characterized in that: The four error compensation probability multipliers combined with the shift circuit and the adder form a 4-bit mixed precision processing unit, which means splitting two 4-bit signed numbers into four 2-bit numbers, which can be expressed as: M=(M1M0) bin =2 2 M1+M0 N = (N1N0) bin = 2 2 N1+N0 Among them, M and N are two input 4-bit signed numbers, M0 and M1 are two 2-bit numbers split by M, N0 and N1 are two 2-bit numbers split by N, (·) bin Represents the value of the binary field, (M1M0) bin Represents the two's complement of the multiplier M, (N1N0) bin Represents the binary complement of the multiplier N, and each coefficient is realized by a shift circuit. The four 2-bit numbers M1, M0, N1, and N0 are respectively input into the four error compensation probability multipliers. The outputs of the four error compensation probability multipliers are input into the shift circuit and summed up by an adder to form a 4-bit mixed precision processing unit. The output of the multiplication of M and N is recorded as K. The 4-bit mixed precision processing unit is expressed by the formula: K=M·N=2 4 (M1·N1) s_sto +2 2 (M1·N0) s_sto +2 2 (M0·N1) s_sto +(M0·N0) u_sto Where K is the product of M and N, (M1·N1) s_sto 、(M1·N0) s_sto 、(M0·N1) s_sto 、(M0·N0) u_sto They are all realized by the error compensation probability multipliers, which are the outputs of the four error compensation probability multipliers, and each coefficient is realized by a shift circuit.
7. The mixed precision hardware acceleration circuit for a deep neural network based on probability calculation according to claim 1, characterized in that: Four of the 4-bit mixed precision processing units are combined with a shift circuit and an adder to form an 8-bit mixed precision processing unit. The two 8-bit signed numbers are recorded as P and Q respectively. The 8-bit number P is decomposed into two 4-bit numbers P1 and P0, and the 8-bit number Q is decomposed into two 4-bit numbers Q1 and Q0, which can be expressed as follows: P=(P1P0) bin Q=(Q1Q0) bin Among them, P and Q are two 8-bit signed multipliers input, P1 and P0 are two 4-bit numbers decomposed from the multiplier P, Q1 and Q0 are two 4-bit numbers decomposed from the multiplier Q, (P1P0) bin is the two's complement of the multiplier P, (Q1Q0) bin is the two's complement of the multiplier Q, Among them, P1 can be further decomposed into two 2-bit numbers P 11 , P 10 , P0 can be further decomposed into two 2-bit numbers P 01 , P 00 , Q1 can be further decomposed into two 2-bit numbers Q 11 , Q 10 , Q0 can be further decomposed into two 2-bit numbers Q 01 , Q 00 , expressed as: P1=(P 11 P 10 ) bin =2 2 P 11 +P 10 P0=(P 01 P 00 ) bin =2 2 P 01 +P 00 Q1=(Q 11 Q 10 ) bin =2 2 Q 11 +Q 10 Q0=(Q 01 Q 00 ) bin =2 2 Q 01 +Q 00 Among them, (P 11 P 10 ) bin is the binary complement of P1, (P 01 P 00 ) bin is the two's complement of P0, (Q 11 Q 10 ) bin is the two's complement of Q1, (Q 01 Q 00 ) bin It is the binary complement of Q0, and each coefficient is realized by the shift circuit. The 2-bit data P after splitting 01 , P 00 , Q 01 , Q 00 The input is sent to the four 4-bit mixed precision processing units, and the outputs are P1Q1, P1Q0, P0Q1, and P0Q0, respectively, which are expressed as follows: P1Q1=2 4 (P 11 Q 11 ) s_sto +2 2 (P 11 Q 10 ) s_sto +2 2 (P 10 Q 11 ) s_sto +2 0 (P 10 Q 10 ) u_sto P1Q0=2 4 (P 11 Q 01 ) s_sto +2 2 (P 11 Q 00 ) s_sto +2 2 (P 10 Q 01 ) u_sto +2 0 (P 10 Q 00 ) u_sto P0Q1=2 4 (P 01 Q 11 ) s_sto +2 2 (P 01 Q 10 ) u_sto +2 2 (P 00 Q 11 ) s_sto +2 0 (P 00 Q 10 ) u_sto P0Q0=2 4 (P 01 Q 01 ) u_sto +2 2 (P 01 Q 00 ) u_sto +2 2 (P 00 Q 01 ) u_sto +2 0 (P 00 Q 00 ) u_sto Among them, P1Q1, P1Q0, P0Q1, P0Q0 are the outputs of the four 4-bit mixed precision processing units, (·) s_sto Represents the probability operation result of a signed number, (·) u_sto represents the probability operation result of an unsigned number, and the items on the right side of the equal sign are the outputs of the error compensation probability multiplier in the 4-bit precision mixing unit after passing through the shift circuit, and the coefficients of each item are realized by the shift circuit. The calculation results P1Q1, P1Q0, P0Q1, and P0Q0 of the four 4-bit mixed precision processing units are input into the adder through the shift circuit, and the calculation results of the 8-bit numbers P and Q are output, which are expressed as follows: P·Q=2 8 P1Q1+2 4 P1Q0+2 4 P0Q1+P0Q0 Among them, each coefficient is realized by a shift circuit.
Citation Information
Patent Citations
Configurable approximate multiplier for quantizing convolutional neural network and implementation method of configurable approximate multiplier
CN110780845A
High-cardinal-number approximate Booth encoding method and mixed-cardinal-number Booth encoding approximate multiplier
CN111488133A
Neural network random number generator sharing circuit, sharing method and processor chip
CN112698811A
Approximate binary multiplier based on static segmentation compensation method
CN115826913A
Neural network random number generator sharing circuit based on random calculation
CN213934855U
Cited By
Random calculation processing unit and method based on adaptive compensation mechanism
CN120653304A
