Storage and calculation integrated chip design method for mixed precision addition neural network

Abstract

The invention discloses a storage and calculation integrated chip design method for a mixed precision addition neural network, and the method comprises the steps: designing a bit-flexible minimum selector circuit which comprises a plurality of encoder circuits, a multi-stage comparison tree circuit and a plurality of selector circuits; the input comprises a weight value, an activation value and a precision configuration signal of the addition neural network, and the output is the minimum value of the weight value and the activation value; a storage and calculation integrated array circuit based on a bit flexible minimum selector circuit is designed; flexible configurable deployment is carried out for addition neural network operators with different precisions, and a configuration signal is set to a bit flexible minimum value selector circuit; and an activation value accumulation result is obtained through an activation summation adder tree circuit, and calculation of an additive neural network operator is completed. According to the method, reasoning calculation acceleration of the addition neural network with mixed precision quantification is supported, the end side operation efficiency can be greatly improved, and the power consumption of end side equipment operation is reduced.

Description

Technical Field

[0001] This invention belongs to the field of integrated circuit design technology, and relates to in-memory computing integrated circuit design technology. Specifically, it relates to an in-memory computing chip design technology for hybrid quantization precision additive neural networks, including circuit structure, operator deployment data flow and its implementation method. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence, the demand for AI chips has also increased significantly. Artificial intelligence is mainly based on neural network model algorithms, such as convolutional neural networks, which are characterized by a large number of model parameters, high computational cost, and primarily matrix multiplication and addition. Especially when deploying neural networks in edge chips, it can cause significant power consumption and performance losses. To reduce the computational overhead of neural networks, recent research has proposed additive neural networks, which significantly reduce computational complexity by converting the multiplication and addition of weights and activation values ​​into absolute value subtraction.

[0003] In-memory computing (CIM) technology significantly reduces data movement and shortens the distance between computation and storage by inserting computing circuitry into the memory array, thereby greatly reducing power consumption and becoming a recent research hotspot. Among these, CIM technology based on static random access memory (SRAM) is particularly suitable for edge-side applications due to its mature process and stable memory nodes. Recent research has also proposed in-memory computing chip designs for additive neural networks, which further simplify absolute value subtraction to a minimum value summation calculation, significantly improving energy efficiency.

[0004] Mixed-precision quantization further reduces the size of model parameters and computational complexity by quantizing different weights and inputs in the same model into different bit widths. However, minimum summation calculations require different computational circuit designs for different precisions. Therefore, current in-memory computing chip designs for additive neural networks are all designed for fixed bit widths and cannot efficiently support mixed-precision calculations. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention provides a method for designing an in-memory computing chip for mixed-precision additive neural networks. This invention designs an in-memory computing array circuit based on a bit-flexible minimum selector circuit and a multi-precision flexible configurable deployment technology. The designed in-memory computing chip can efficiently support the computation of 2-bit, 4-bit, and 8-bit precision additive neural networks, and has high circuit utilization and energy efficiency.

[0006] The technical solution of this invention is: A method for designing an in-memory computing chip for mixed-precision additive neural networks: Based on conventional in-memory computing (CIM) systems, this invention improves upon existing CIMs by designing an in-memory computing array circuit based on a bit-flexible minimum selector circuit. This circuit efficiently supports 2-bit, 4-bit, and 8-bit precision additive neural networks. Furthermore, this invention employs a multi-precision, flexible, and configurable deployment technology. For 2-bit, 4-bit, and 8-bit precision additive neural networks, the circuit can be flexibly configured to achieve efficient operator deployment and computation, significantly improving the computational efficiency of edge-side additive neural networks.

[0007] In specific implementation, the in-memory array circuit based on the bit-flexible minimum selector circuit designed in this invention includes a read / write circuit, an address decoder circuit, multiple sub-blocks (banks), an activation summation adder tree circuit (summing the activation values), and multiple result generation circuits. Each bank contains multiple static random access memories (SRAMs), multiple bit-flexible minimum selector circuits, and one adder tree (summing the results of multiple bit-flexible minimum selector circuits). The SRAM read / write circuit is responsible for reading and writing weight data in the SRAM within the bank; the bank is responsible for storing the weights W and weight summation data Wsum of the additive neural network, comparing the weights W and the input activation values ​​ACT to obtain the minimum value, and summing the minimum values ​​to obtain the sum of the minimum values ​​Csum; the activation summation adder tree circuit is responsible for summing and accumulating the activation values ​​of the input additive neural network to obtain the activation value summation data ACTsum; the result generation circuit is used to generate the final calculation result based on the operators of the additive neural network. The bit-flexible minimum selector circuit designed in this invention consists of 8 encoder circuits, 1 multi-level comparison tree circuit, and 8 selector circuits. The input to the bit-flexible minimum selector circuit includes an 8-bit weight W, an 8-bit activation value ACT, and a precision configuration signal (used to configure the selection of different bits). The output is the minimum value between the weight and the activation value. The encoder circuit in the bit-flexible minimum selector circuit performs digital logic operations on the weight and activation value, converting them into a generation signal G and a propagation signal P. Then, a multi-level comparison tree circuit compares the G and P signals in stages, obtaining comparison signals for 2 bits, 4 bits, and 8 bits respectively. The bit-flexible minimum selector circuit first selects the corresponding bit comparison signal based on the precision configuration signal, and then selects the minimum value between the weight and activation values ​​of the additive neural network based on the comparison signal, outputting it to the addition tree. Ultimately, it can achieve minimum value selection and summation calculation at different precisions.

[0008] In the specific deployment of the additive neural network operator, the SRAM in each bank of the in-memory computing array circuit is used to store the neural network's weight data W and the weight summation data Wsum. The activation value data is input as an input vector into the in-memory computing array circuit and calculated with the weights. For additive neural networks of different precisions (e.g., 2-bit, 4-bit, and 8-bit), the deployment scheme can be flexibly configured. For example, for an N-bit (N is 2, 4, or 8) additive neural network, one bank contains M-bit SRAM cells. That is, each bank stores M / N (M divided by N) weight data, and M / N activation data are input for each calculation, with each weight data corresponding to one activation data. Then, the bit-flexible minimum value selector circuit is configured for M-bit precision calculation. In the bit-flexible minimum value selector circuit, the weight data and activation value data are compared and calculated to obtain M / N minimum value data, which are then sent to subsequent circuits to implement the entire operator's calculation.

[0009] During computation, the activation value data first passes through an activation summation adder tree circuit to obtain the activation value summation data ACTsum. Then, each bank within the in-memory computing array reads multiple weight data stored in SRAM. According to the adder network operator, each weight data corresponds to an activation data. The weight data and activation value data are fed together into the bit-flexible minimum value selector circuit and the adder tree circuit within the bank. The bit-flexible minimum value selector circuit selects the M-bit comparison signal from the multi-level comparison tree circuit based on the precision configuration signal, and then selects the minimum value of the weight and activation based on the comparison signal, and performs minimum value selection summation to obtain the minimum value sum Csum. The weight summation data Wsum, the activation value summation data ACTsum, and the minimum value sum Csum are fed together into the result generation circuit to calculate ACTsum + Wsum - 2 × Csum, obtaining the calculation result PSUM of the adder neural network operator, which is the activation value data output by this layer. After subsequent activation function calculations, PSUM will continue to be passed to the next layer as the activation value data input for the next layer.

[0010] Compared with the prior art, the beneficial technical effects of the present invention are as follows: This invention provides a method for designing an in-memory computing chip for mixed-precision additive neural networks. It involves designing a bit-flexible minimum selector circuit and an in-memory computing array circuit based on the bit-flexible minimum selector circuit. The bit-flexible minimum selector circuit includes multiple encoder circuits, a multi-level comparison tree circuit, and multiple selector circuits. The input to the bit-flexible minimum selector circuit includes the weight values, activation values, and precision configuration signals of the additive neural network. The output is the minimum value among the weight values ​​and activation values. Flexible and configurable deployment is implemented for additive neural network operators of different precisions, and configuration signals are set for the bit-flexible minimum selector circuit. The activation values ​​are then accumulated through an activation summation addition tree circuit, completing the computation of the additive neural network operator. This invention can flexibly support the deployment and computation of additive network operators under different bit precisions and achieve 100% hardware utilization for different bit precisions. This enables accelerated inference computation for mixed-precision quantized additive neural networks, significantly improving the efficiency of edge operation and reducing the power consumption of edge devices. Attached Figure Description

[0011] Figure 1 This is a flowchart of the method of the present invention.

[0012] Figure 2 This is a schematic diagram of the architecture of an in-memory computing array circuit.

[0013] Figure 3 This is a schematic diagram of the architecture of a bit-flexible minimum selector circuit.

[0014] Figure 4 This is a schematic diagram illustrating the working principle of a bit-flexible minimum selector circuit at different precision levels. Detailed Implementation

[0015] The present invention will be further illustrated below with reference to the accompanying drawings and embodiments, but the scope of the invention is not limited in any way.

[0016] In practical implementation, the in-memory computing chip design method proposed in this invention can significantly improve the inference computation efficiency of mixed-precision quantized additive neural networks, and can efficiently support additive neural network computations of different precisions: 2-bit, 4-bit, and 8-bit. The in-memory computing chip design method for mixed-precision additive neural networks proposed in this invention will be described in detail below, such as... Figure 1 As shown, the method of the present invention includes the following steps: 1) Design of an in-memory computing array circuit based on a bit-flexible minimum selector circuit: This includes a read / write circuit, multiple sub-banks, an address decoder circuit, an activation summation adder tree circuit, and multiple result generation circuits. Each bank contains multiple SRAMs, multiple bit-flexible minimum selector circuits, and one adder tree circuit. The innovative design of this invention is the design of the bit-flexible minimum selector circuit and the in-memory computing array circuit based on it. This in-memory computing array circuit can support the deployment and computation of additive neural network operators with different precisions, significantly improving the inference computation efficiency of additive neural networks after mixed-precision quantization.

[0017] 2) Flexible and configurable deployment of additive neural network operators with different precisions: For additive neural network operators with different precisions, the weights corresponding to the bit width precision are stored in the SRAM of the in-memory computing array circuit, the activation corresponding to the bit width precision is input into the in-memory computing array circuit, and the configuration signal is set to the bit flexible minimum value selector circuit according to the precision of the current operator.

[0018] 3) Completion of the additive neural network operator calculation: The activation value data is processed by the activation summation addition tree circuit to obtain the activation value summation data ACTsum. Each bank within the in-memory array reads the weights W and weight summation data Wsum from the SRAM of the additive neural network. The weight data W and the activation value data ACT are fed together into the bit-flexible minimum value selector circuit and the addition tree circuit within the bank. The bit-flexible minimum value selector circuit selects the corresponding bit comparison signal according to the precision configuration signal, and then selects the minimum value of the weights and activations according to the comparison signal, performing minimum value selection and summation calculation to obtain the minimum value sum Csum. Wsum, ACTsum, and Csum are then fed together into the result generation circuit to calculate ACTsum + Wsum - 2 × Csum, thus obtaining the calculation result PSUM of the additive neural network operator.

[0019] In specific implementation, this invention designs a memory-computing integrated array circuit based on a bit-flexible minimum selector circuit. For example... Figure 2As shown, it includes one read / write circuit, 32 sub-blocks (banks), one address decoder circuit, one activation summation adder tree circuit, and 32 result generation circuits. Each bank contains 256 SRAMs, 32 bit-flexible minimum selector circuits, and one adder tree. The SRAM read / write circuit is responsible for reading and writing data in the SRAM within the bank; the bank is responsible for storing the weights W and the weight summation data Wsum, and for performing minimum value selection and summation calculation to obtain the minimum sum Csum; the activation summation adder tree circuit is responsible for summing and accumulating the input activation values ​​to obtain the activation value summation data ACTsum; the result generation circuit generates the final calculation result PSUM based on the operators of the addition neural network.

[0020] Bit-flexible minimum selector circuit such as Figure 3 As shown, its inputs are an 8-bit weight value W<0:7>, an 8-bit weight logical inverted value WB<0:7>, and an 8-bit activation inverted value XB<0:7>. The bit-flexible minimum value selector circuit consists of 8 encoder circuits, 1 three-level comparison tree circuit, and 8 selector circuits. Each encoder performs a logical AND operation on 1 bit W and 1 bit XB to obtain a generated signal G, and performs a logical XOR operation on 1 bit WB and 1 bit XB to obtain a propagation signal P. Finally, the 8 encoder circuits generate an 8-bit generated signal G<0:7> and an 8-bit propagation signal P<0:7>. The three-level comparison tree circuit compares the G<0:7> and P<0:7> signals in stages to obtain comparison signals at 2 bits, 4 bits, and 8 bits, respectively. The 8 selector circuits first select the corresponding bit comparison signal according to the precision configuration signal, and then select the minimum value OUT<0:7> of weight and activation according to the comparison signal, and output it to the addition tree. Finally, it can realize minimum value selection and summation calculation at different precisions. The bit-flexible minimum selector can be flexibly configured according to different bit precisions. For example... Figure 4 As shown, for a 2-bit additive neural network, configuring the bit-flexible minimum selector circuit for 2-bit precision computation allows each circuit to perform four sets of 2-bit computations. For a 4-bit additive neural network, configuring the bit-flexible minimum selector circuit for 4-bit precision computation allows each circuit to perform two sets of 4-bit computations. For an 8-bit additive neural network, configuring the bit-flexible minimum selector circuit for 8-bit precision computation allows each circuit to perform one set of 8-bit computations.

[0021] In the specific deployment of additive neural network operators, the SRAM in the in-memory computing unit is used to store the neural network's weight data W and the weight summation data Wsum. Activation value data is input as an input vector into the in-memory computing array to perform calculations with the weights. For additive neural networks of different precisions (2-bit, 4-bit, and 8-bit), the deployment scheme can be flexibly configured. For example, for a 2-bit additive neural network, each bank stores 128 weight data points, and 128 activation data points are input for each calculation, with each weight data point corresponding to one activation data point. Then, the bit-flexible minimum value selector circuit is configured for 2-bit precision calculation. Each bit-flexible minimum value selector circuit can perform four sets of 2-bit calculations, and each bank can perform minimum value selection and summation calculations for 128 weight data points and 128 activation data points. For a 4-bit additive neural network, each bank stores 64 weight data points, and 64 4-bit activation data points are input for each calculation, with each weight data point corresponding to one activation data point. Then, the bit-flexible minimum selector circuit is configured for 4-bit precision computation. Each bit-flexible minimum selector circuit can perform two sets of 4-bit computations, and each bank can perform minimum selection and summation computations on 64 weight data and 64 activation data. For an 8-bit additive neural network, each bank stores 32 weight data, and 32 8-bit activation data are fed in for each computation, with each weight data corresponding to one activation data. Then, the bit-flexible minimum selector circuit is configured for 8-bit precision computation. Each bit-flexible minimum selector circuit can perform one set of 8-bit computations, and each bank can perform minimum selection and summation computations on 32 weight data and 32 activation data.

[0022] During calculation, the activation value data first passes through the activation summation adder tree circuit to obtain the activation value sum data ACTsum. Then, each bank within the in-memory array reads multiple weight data stored in SRAM. According to the adder network operator, each weight data corresponds to an activation data. The weight data and activation value data are fed together into the bit-flexible minimum value selector circuit and adder tree circuit within the bank. The selector circuit selects an 8-bit comparison signal from the multi-level comparison tree based on the precision configuration signal, and then selects the minimum value of the weight data W and the activation value data ACT based on the comparison signal, and performs minimum value selection summation to obtain the minimum value sum Csum. Wsum, ACTsum, and Csum are then fed together into the result generation circuit to calculate ACTsum + Wsum - 2 × Csum, which yields the calculation result PSUM of the adder neural network operator.

[0023] It should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the scope of the claims.

Claims

1. A method for a memory computing chip design for a mixed-precision addition neural network, the method comprising: Comprising the following steps: 1) Designing a bit-flexible minimum selector circuit and a memory-compute integrated array circuit based on the bit-flexible minimum selector circuit; 11) The bit-flexible minimum selector circuit comprises a plurality of encoder circuits, a multi-stage comparison tree circuit, and a plurality of selector circuits; The inputs of the bit-flexible minimum selector circuit include the weight values, activation values, and precision configuration signals of the addition neural network, and the output is the minimum value among the weight values and activation values; The encoder circuit in the bit-flexible minimum selector circuit performs digital logic operations on the weight and activation values to convert them into generation signals and propagation signals; the multi-stage comparison tree circuit compares the generation signals and propagation signals in stages to obtain comparison signals corresponding to the bits; According to the precision configuration signal, the comparison signal corresponding to the bit is selected, and then the minimum value of the weight value and the activation value of the addition neural network is selected according to the comparison signal and output to the addition tree to realize minimum value selection and summation calculation under different precisions; 12) Designing a memory-compute integrated array circuit based on the bit-flexible minimum selector circuit; The memory-compute integrated array circuit comprises a read-write circuit, a sub-block bank, an address decoder circuit, an activation summation addition tree circuit, and a result generation circuit; each bank contains a plurality of static random access memories (SRAMs), a plurality of bit-flexible minimum selector circuits, and an addition tree; the read-write circuit is used for reading and writing data in the SRAMs in the bank; the bank is used for storing weights and weight summation data and realizing minimum value selection and summation calculation; the activation summation addition tree circuit is used for summing the input activation values; and the result generation circuit is used for generating a calculation result according to the operators of the addition neural network; 2) Flexible and configurable deployment for different precision addition neural network operators: For different precision addition neural network operators, store the weights of corresponding bit width precision in the SRAMs in the banks of the memory-compute integrated array circuit, input the activation values of corresponding bit width precision into the memory-compute integrated array circuit, and set the configuration signal to the bit-flexible minimum value selector circuit according to the precision of the current operator; 3) Obtain the activation value summation data through the activation summation addition tree circuit to complete the calculation of the addition neural network operator; Each bank in the memory-compute integrated array reads the weight values and weight summation data in the SRAMs; Send the weight values and activation values together into the bit-flexible minimum value selector circuit and the addition tree circuit in the bank; the bit-flexible minimum value selector circuit selects the comparison signal corresponding to the bit according to the precision configuration signal, then selects the minimum value of the weight value and the activation value according to the comparison signal, and performs minimum value selection and summation calculation to obtain the sum of the minimum values; Send the weight summation data Wsum, the activation value summation data ACTsum, and the sum of the minimum values Csum together into the result generation circuit for calculation, and obtain the activation value result of the addition neural network operator output.

2. The design method of the integrated chip of storage and calculation for the mixed precision addition neural network according to claim 1, wherein, In step 3), the calculation in the result generation circuit is: ACTsum+Wsum-2×Csum, which is the calculation result of the addition neural network operator.

3. The design method of the integrated chip of storage and calculation for the mixed precision addition neural network according to claim 1, wherein, The bit flexible minimum selector circuit is composed of 8 encoder circuits, 1 multi-stage comparison tree circuit and 8 selector circuits.

4. The design method of the integrated chip of storage and calculation for the mixed precision addition neural network according to claim 3, wherein, The input of the bit flexible minimum selector circuit includes 8-bit weight W, 8-bit activation value ACT and different bit precision configuration signals.

5. The design method of the integrated chip of storage and calculation for the mixed precision addition neural network according to claim 1, wherein, The bit flexible minimum selector circuit based on the storage calculation integrated array circuit includes 1 read-write circuit, 32 sub-banks, 1 address decoder circuit, 1 activation summation addition tree circuit and 32 result generation circuits; each bank contains 256 SRAMs, 32 bit flexible minimum selector circuits and 1 addition tree.

6. The design method of an integrated chip for a mixed-precision addition neural network according to claim 1, wherein The bit flexible minimum selector is flexibly configured according to different bit precisions.

7. The design method of the integrated chip of storage and calculation for the mixed precision addition neural network according to claim 6, wherein, For a 2-bit addition neural network, the bit flexible minimum selector circuit is configured to calculate at a precision of 2 bits, and each bit flexible minimum selector circuit realizes 4 groups of 2-bit calculation.

8. The design method of the integrated chip of storage and calculation for the mixed precision addition neural network according to claim 6, wherein, For a 4-bit addition neural network, the bit flexible minimum selector circuit is configured to calculate at a precision of 4 bits, and each bit flexible minimum selector circuit realizes 2 groups of 4-bit calculation.

9. The design method of the integrated chip of storage and calculation for the mixed precision addition neural network according to claim 6, wherein, For an 8-bit addition neural network, the bit flexible minimum selector circuit is configured to calculate at a precision of 8 bits, and each bit flexible minimum selector circuit realizes 1 group of 8-bit calculation.

Images