Energy-efficient binary neural network accelerator for AI IoT

Abstract

The present invention provides a high-efficiency binary neural network accelerator for artificial intelligence (AI) and the Internet of Things (IoT). The present invention provides a 0.3-0.6V sub- / near-threshold 10T1C multiplication bit cell with series capacitors for charge-domain binary convolution; a process-resistant differential voltage amplifier array between the bit line and the DAC for robust pre-amplification during 0.3V batch normalization; and an inert bit line reset scheme to further reduce energy with negligible loss in inference accuracy. Thus, the binary neural network accelerator chip based on in-memory computation achieves peak energy efficiencies of 18.5 POPS / W and 6.06 POPS / W, respectively, representing improvements of 21× and 135× over previous macro and system work [9,11].

Description

Technical Field

[0001] The present invention relates to a 0.3V high-energy-efficiency binary neural network accelerator. Background Art

[0002] Artificial Intelligence Internet of Things (AIoT) is the fusion of artificial intelligence technology (AI) and the Internet of Things (IoT) in practical applications. With its powerful application scenarios, AI IoT has become the most likely and plastic development direction in the field of artificial intelligence, and is also the best way to upgrade traditional industries to intelligent ones. AI IoT applies AI capabilities to IoT infrastructure for ubiquitous data analysis. Therefore, due to the scarce resources and limited energy budget of distributed IoT platforms, energy-efficient AI solutions are very popular. To this end, quantized convolutional neural networks (QCNNs) are widely regarded as a promising technology for AIoT applications [1,2] with simplified model size, computation, and energy. In particular, [3] has proposed a binarized convolutional neural network (BCNN), in which pre-trained weights are actively quantized to 1 bit, which can save a lot of hardware and energy while providing satisfactory computational accuracy for AIoT reasoning tasks [4]. Although BCNNs [5-7] have inherent algorithmic advantages, optimized hardware architectures such as near memory computing (NMC) and in-memory computing (IMC) with analog domain computing are also actively seeking to minimize the energy of data movement from on-chip memories [8,9]. In terms of memory type used, recent eDRAM designs have achieved higher macro-computing density, but their complex bit cell refresh modules limit their achievable peak energy efficiency [10,11]. Compared to recent multi-bit SRAM work [12,13], the current state-of-the-art (SOTA) [9] charge domain memory (CIM) BNN accelerator achieves the best energy efficiency to date.

[0003] However, existing binary neural network accelerators using charge domain in-memory computing still face the following challenges in improving energy efficiency:

[0004] (1) The bit line is connected to a large number of parallel capacitors, which consumes a lot of convolution energy;

[0005] (2) Working at subthreshold / near-threshold voltages can improve the energy efficiency of binary neural network accelerators, but it will cause severe accuracy loss;

[0006] (3) The most advanced structure of the bit line inevitably requires the constant resetting of the bit line after each convolution, which hinders the further reduction of the bit line energy consumption.

[0007] References:

[0008] [1]Z.Liu,E.Ren,F.Qiao,Q.Wei,X.Liu,L.Luo,H.Zhao,and H.Yang,“NS-CIM:ACurrent-Mode Computation-in-Memory Architecture Enabling Near-SensorProcessing for Intelligent IoT Vision Nodes,”IEEE Trans.Circuits Syst.I,vol.67,no.9,pp.2909–2922,2020.

[0009] [2]A.Di Mauro,F.Conti,P.D.Schiavone,D.Rossi,and L.Benini,“Always-On674μW@4GOP / s Error Resilient Binary Neural Networks With Aggressive SRAMVoltage Scaling on a 22-nm IoT End-Node,”IEEE Trans.Circuits Syst.I,vol.67,no.11,pp.3905–3918,2020.

[0010] [3]M.Courbariaux,I.Hubara,D.Soudry,R.El-Yaniv,and Y.Bengio,“Binarizedneural networks:Training deep neural networks with weights and activationsconstrained to+1 or-1,”arXiv preprint arXiv:1602.02830,2016.

[0011] [4]B.Moons,K.Goetschalckx,N.Van Berckelaer,and M.Verhels,“Minimumenergy quantized neural networks,”in 2017 51st Asilomar Conf.on Signals,Systems,and Computers.IEEE,2017,pp.1921–1925.

[0012] [5]S.Zheng,P.Ouyang,D.Song,X.Li,L.Liu,S.Wei,and S.Yin,“An ultra-lowpower binarized convolutional neural network-based speech recognitionprocessor with on-chip self-learning,”IEEE Trans.Circuits Syst.I,vol.66,no.12,pp.4648–4661,2019.

[0013] [6]Y.Li,Z.Liu,W.Liu,Y.Jiang,Y.Wang,W.L.Goh,H.Yu,and F.Ren,“A 34-FPS698-GOP / s / W binarized deep neural network-based natural scene textinterpretation accelerator for mobile edge computing,”IEEETrans.Ind.Electron.,vol.66,no.9,pp.7407–7416,2018.

[0014] [7]M.Koo,G.Srinivasan,Y.Shim,and K.Roy,“SBSNN:Stochastic-bits enabledbinary spiking neural network with on-chip learning for energy efficientneuromorphic computing at the edge,”IEEE Trans.Circuits Syst.I,vol.67,no.8,pp.2546–2555,2020.

[0015] [8]D.Bankman,L.Yang,B.Moons,M.Verhelst,and B.Murmann,“An Always-On3.8μJ / 86%CIFAR-10 Mixed-Signal Binary CNN Processor With All Memory on Chipin 28-nm CMOS,”IEEE J.Solid-State Circuits,vol.54,no.1,pp.158–172,2018.

[0016] [9]H.Valavi,P.J.Ramadge,E.Nestler,and N.Verma,“A 64-tile 2.4-mb in-memory-computing cnn accelerator employing charge-domain compute,”IEEEJ.Solid-State Circuits,vol.54,no.6,pp.1789–1799,2019.

[0017]

[10] S.Xie et al.,"16.2eDRAM-CIM:Compute-In-Memory Design withReconfigurable EmbeddedDynamic-Memory Array Realizing Adaptive DataConverters and Charge-Domain Computing,"ISSCC,pp.248-250,2021.

[0018]

[11] Z.Chen et al.,"15.3A 65nm 3T Dynamic Analog RAM-Based Computing-in-Memory Macro and CNN Accelerator with Retention Enhancement,AdaptiveAnalog Sparsity and 44TOPS / W System Energy Efficiency,"ISSCC,pp.240-242,2021.

[0019]

[12] J.Yue et al.,"14.3A 65nm Computing-in-Memory-Based CNN Processorwith 2.9-to-35.8TOPS / W System Energy Efficiency Using Dynamic-SparsityPerformance-Scaling Architecture and EnergyEfficient Inter / Intra-Macro DataReuse,"ISSCC,pp.234-236,2020.

[0020]

[13] Q.Dong et al., "A 351TOPS / W and 372.4GOPS Compute-in-Memory SRAMMacro in 7nm FinFET CMOS for Machine-Learning Applications", ISSCC, pp.242-243, 2020. Summary of the Invention

[0021] The purpose of the present invention is to solve the problem of improving energy efficiency of the existing binary neural network accelerator for charge domain in-memory computing.

[0022] In order to achieve the above objectives, the technical solution of the present invention is to provide a high-energy-efficiency binary neural network accelerator that can be used in artificial intelligence Internet of Things, characterized by comprising:

[0023] A multiplication bit array, which consists of L×L sub- / near-threshold 10T1C multiplication bit cells based on series capacitors; the sub- / near-threshold 10T1C multiplication bit cell is used to perform binary multiplication and accumulation operations within one clock cycle, and consists of a 6T storage bit cell and a 4T1C memory XNOR logic cell, wherein: the 6T storage bit cell is used to store weight values ​​W and WB, W is the weight value of the pre-trained neural network, and WB is the negation of W; the 4T1C memory XNOR logic cell implements the XNOR logic between the input activation values ​​F and FB and the weights W and WB stored in the 6T storage bit cell, the capacitor in the 4T1C memory XNOR logic cell is connected in series to the accumulation bit line ABL, and the multiplication result is directly generated on the accumulation bit line ABL through charge distribution of the capacitor of the 4T1C memory XNOR logic cell to generate the final convolution result, wherein F is the binary value of the input activation value and FB is the negation of F;

[0024] A 20×L voltage amplifier array is designed with 20 low-voltage amplifier units of different transistor sizes in each column. Based on the pre-trained binary batch normalization bias coefficient α, only one low-voltage amplifier unit is selected from the 20 low-voltage amplifier units in each column for correct amplification. The selected low-voltage amplifier unit is configured with a corresponding body bias voltage, and the output voltage is close to 1 / 2VDD to ensure that each low-voltage amplifier unit obtains maximum voltage gain.

[0025] Preferably, the sub- / near-threshold 10T1C multiplication bit cell adopts a multi-VT design strategy to overcome the PMOS / NMOS skew at low voltage.

[0026] Preferably, the PMOS transistor used in the 6T storage bit cell is LVT, and the NMOS transistor is HVT.

[0027] Preferably, all transistors of the sub- / near-threshold 10T1C multiplication bit cell adopt gate length biasing technology to reduce the influence of device mismatch, and adopt adaptive body biasing technology to reduce device deviation.

[0028] Preferably, the capacitor of the 4T1C memory XNOR logic unit is implemented by a MOM capacitor with high-level metal interposers.

[0029] Preferably, the voltage amplification array realizes amplification based on the following steps:

[0030] 1) Select a low-voltage amplifier unit with the largest gain from the 20 low-voltage amplifier units in each column according to the bias coefficient α;

[0031] 2) The bias coefficient α is converted into an analog voltage value V1 through the DAC unit and input to the input of the low-voltage amplifier unit selected in the previous step. The output voltage V1' of the low-voltage amplifier unit is sampled and held using the first sampling capacitor;

[0032] 3) The analog voltage value V2 on the accumulated bit line ABL is sent to the input terminal of the same low-voltage amplifier unit in step 2), and the output voltage V2' of the low-voltage amplifier unit is sampled and held using a second sampling capacitor;

[0033] 4) Since V2'-V1'>>V2-V1, the voltages output by the first sampling capacitor and the second sampling capacitor are passed through the low-voltage comparator to obtain a correct comparison result, that is, a batch normalization result is obtained.

[0034] Preferably, determining the optimal body bias voltages of all selected low-voltage amplifying units by a body bias voltage calculation circuit includes the following steps:

[0035] a) For each column of low-voltage amplifier units, 20 pre-sized low-voltage amplifier units are input with equally spaced scan voltages ranging from 1 / 21VDD to 20 / 21VDD one by one;

[0036] b) The outputs of 20 low-voltage amplifier units with input values ​​ranging from 1 / 21VDD to 20 / 21VDD are continuously compared with a reference voltage of 1 / 2VDD via an operational amplifier (OPA). The output of the op amp (OPA) uses a closed-loop method to body-bias all NMOS transistors in the current low-voltage amplifier unit. The output body-bias voltage is further digitized by a 6-bit SAR ADC unit to obtain a digital body-bias code for the current low-voltage amplifier unit. This digital body-bias code is then stored in a 6-bit register corresponding to each low-voltage amplifier unit.

[0037] c) After selecting a low-voltage amplifier unit with the largest gain from the 20 low-voltage amplifier units in each column based on the bias coefficient α, the body bias digital code corresponding to the selected low-voltage amplifier unit is retrieved from the register, restored to an analog value through the DAC unit, and input into the body bias node of the NMOS transistor of the corresponding low-voltage amplifier unit, thereby starting voltage amplification of the bias coefficient α and the accumulation bit line ABL.

[0038] Preferably, the DAC unit adopts a 5-bit auxiliary DAC unit and a 6-bit high-precision DAC unit based on a capacitor array, wherein the 5-bit auxiliary DAC unit pre-charges the parasitic capacitance of the input node and the body bias node of the low-voltage amplifier unit to generate a roughly estimated analog voltage, and the 6-bit high-precision DAC unit generates an accurate analog voltage on this basis.

[0039] Preferably, the accumulation bit line ABL adopts an inert accumulation bit line reset scheme, which sets the reset interval N according to different power supply voltages and the convolution kernel size adopted by the neural network. The reset operation of the accumulation bit line ABL takes effect after the Nth convolution and batch normalization operation.

[0040] Compared with the existing technology, the binary neural network accelerator proposed in this invention has the following characteristics:

[0041] (1) 0.3-0.6V sub- / near-threshold 10T1C multiplication bit cell with series capacitor for charge-domain binary convolution;

[0042] (2) The present invention provides a process-resistant, differential voltage amplifier array between bit lines and DACs for robust pre-amplification in batch-normalized operation at 0.3V;

[0043] (3) The present invention provides an inert bit line reset scheme to further reduce energy and infer that the loss of accuracy is negligible.

[0044] Thus, the binary neural network accelerator chip based on in-memory computing provided by the present invention achieves a peak energy efficiency of 18.5 POPS / W and 6.06 POPS / W, which are 21× and 135× higher than previous macro and system works [9,11], respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1A 、 Figure 1B 、 Figure 1C and Figure 1D The comparison between the present invention and the prior art is illustrated, wherein: Figure 1A This shows that the present invention solves the problem that the existing technology requires too much energy for line convolution. Figure 1B This illustrates the problem of near / sub-threshold accuracy loss in the prior art of the present invention. Figure 1C This shows that the present invention solves the problem in the prior art that the accumulation bit line needs to be reset after each convolution. Figure 1D The peak energy efficiency comparison between the present invention and the prior art is illustrated;

[0046] Figure 2 This is a system architecture diagram of the present invention;

[0047] Figure 3 Schematic diagram of a 0.3-0.6V sub / near-threshold 10T1C multiplication bit cell;

[0048] Figure 4 The multiplication operation table of the 10T1C multiplication bit unit;

[0049] Figure 5 A comparison diagram of the series capacitor solution of the present invention and the existing parallel capacitor solution;

[0050] Figure 6 Schematic diagram of voltage amplification array;

[0051] Figure 7 This is a schematic diagram of the working process of the voltage amplification array;

[0052] Figure 8 Schematic diagram of DAC unit;

[0053] Figure 9 This is a schematic diagram of the adjustment effect of the high-gain amplifier array in the present invention;

[0054] Figure 10 Schematic diagram of the lazy ABL reset (LAR) operating mechanism of the present invention;

[0055] Figure 11 Comparison of the LAR reset intervals N = 1 and N = 2;

[0056] Figure 12 The reset interval values ​​of LAR corresponding to different convolution kernels under 0.3V-0.6V power supply;

[0057] Figure 13 This is a comparison chart of energy consumption after adopting LAR;

[0058] Figure 14 The measurement results are shown in Figure 2.

[0059] Figure 15 A comparison table of the present invention and the prior art. DETAILED DESCRIPTION

[0060] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0061] like Figure 2 As shown, the present invention provides a 0.3V high-energy-efficiency binary neural network accelerator for application in artificial intelligence Internet of Things, including a multiplication bit array, which is composed of 64×64 0.3-0.6V sub / near-threshold 10T1C multiplication bit units based on series capacitors.

[0062] Figure 3 The design of a 10T1C multiplication bit unit for energy-optimized and reliable charge domain binary convolution operations when the power supply voltage drops to 0.3-0.6V is shown. The multiplication bit unit in the present invention is used to perform binary multiplication and accumulation operations within one clock cycle and consists of a 6T storage bit unit M1-M6 and a 4T1C memory XNOR logic unit M7-M10, C. The 6T storage bit unit is used to store weight values ​​W and WB. The 4T1C memory XNOR logic unit implements the XNOR logic between the input activation values ​​F and FB and the weights W and WB stored in the 6T storage bit unit. The MOM capacitor in the 4T1C memory XNOR logic unit is connected in series to the accumulation bit line ABL. In the above content, F is the binary value of the input activation value, FB is the negation of F, W is the weight value of the pre-trained neural network, and WB is the negation of W.

[0063] When writing the weight values ​​W and WB into the 6T storage bit cells, the write channels of the bit lines BL and BLB are opened row by row through the signal on the word line WL (BLB is the negation of BL). The pre-trained weight values ​​W and WB are written into the 6T storage bit cells of the 10T1C multiplication bit cells through the bit lines BL and BLB respectively. After the activation values ​​F and FB are input from outside the chip, the weight values ​​W and WB stored in the 6T storage bit cells are multiplied by 1 bit with the activation values ​​F and FB through the 4T1C memory XNOR logic unit. The multiplication result is directly generated on the accumulation bit line ABL through the charge distribution of the MOM capacitor of the 4T1C memory XNOR logic unit to generate the final convolution result.

[0064] Multiplication in a binary neural network uses two values, +1 and -1, which map to logical "1" and "0" in hardware.

[0065] Combine Figure 4 , 10T1C multiplication operation example:

[0066] (1) The input activation value is F=1, FB=0 (i.e., the input is +1), and the stored weight value is W=1, WB=0 (i.e., the weight is +1). The result is F⊕W=1 (i.e., the result is +1).

[0067] (2) The input activation value is F=1, FB=0 (i.e., the input is +1), and the stored weight value is W=0, WB=1 (i.e., the weight is -1), and the result is F⊕W=0 (i.e., the result is -1).

[0068] (3) The input activation value is F=0, FB=1 (i.e., the input is -1), and the stored weight value is W=1, WB=0 (i.e., the weight is +1), and the result is F⊕W=0 (i.e., the result is -1).

[0069] (4) The input activation value is F=0, FB=1 (i.e., the input is -1), and the stored weight value is W=0, WB=1 (i.e., the weight is -1), and the result is F⊕W=1 (i.e., the result is +1).

[0070] Accumulated bit line ABL operation example:

[0071] (1) If 64 capacitors are connected to the accumulation bit line ABL, where the number of capacitors with F⊕W=1 is 64 and the number of capacitors with F⊕W=0 is 0, then the accumulation bit line obtains the convolution result of VDD.

[0072] (2) If 64 capacitors are connected to the cumulative bit line, of which the number of capacitors with F⊕W=1 is 0 and the number of capacitors with F⊕W=0 is 64, then the cumulative bit line obtains the convolution result of GND.

[0073] (3) If 64 capacitors are connected to the cumulative bit line, of which 32 capacitors have F⊕W=1 and 32 capacitors have F⊕W=0, then the cumulative bit line obtains a convolution result of 1 / 2VDD.

[0074] (4) If 64 capacitors are connected to the cumulative bit line, where the number of capacitors with F⊕W=1 is Np and the number of capacitors with F⊕W=0 is 64-Np, then the cumulative bit line obtains the convolution result of (Np / 64)*VDD.

[0075] Compared with the previous bit cell based on parallel capacitors [9], the 0.3-0.6V sub- / near-threshold 10T1C multiplication bit cell based on series capacitors proposed in this invention saves a lot of energy. The equivalent charge domain circuit model is as follows Figure 5As shown. In the bit cell design based on parallel capacitors, the final accumulation is achieved by charge redistribution between all parallel capacitors and parasitic diffusion capacitances. Its energy consumption is proportional to the number of charged capacitors along the cumulative bit line. In contrast, in the 0.3-0.6V sub / near-threshold 10T1C multiplication bit cell design based on series capacitors proposed in the present invention, a different charge domain process is adopted, and the accumulated voltage on the cumulative bit line is composed of the number of charged capacitors (i.e., the number of F⊕W=1) and the number of non-charged capacitors (i.e., the number of F⊕W=0). Assuming that the number of charged capacitors (i.e., the number of XNOR=1) is uniformly distributed, the present invention theoretically reduces the convolution energy by 66.7% compared with the existing parallel capacitor-based design.

[0076] In addition, the 0.3-0.6V sub- / near-threshold 10T1C multiplication bit cell based on series capacitors provided by the present invention is further optimized in the following aspects to ensure low-voltage reliability under process deviations and reduce standby power.

[0077] The 10T1C multiplication bit cell proposed in the present invention adopts a multi-VT design strategy to overcome the PMOS / NMOS bias at low voltage, wherein the PMOS transistor used in the 6T storage bit cell is LVT and the NMOS transistor is HVT. In addition, gate length bias technology is used in all transistors (LN=80nm, LP=150nm) to reduce the impact of device mismatch, and adaptive body bias technology is used to reduce device deviation. In order to save area, the capacitor of the 4T1C memory XNOR logic unit is implemented using a MOM capacitor with high-level interdigital metal. The area of ​​the 10T1C multiplication bit cell proposed in the present invention is only 1.98 times that of the 6T bit cell.

[0078] like Figure 6 As shown, the energy-efficient binary neural network accelerator provided by the present invention also includes a 20×64 voltage amplifier array, wherein each column is designed with 20 low-voltage amplifier units of different transistor sizes. However, based on the pre-trained binary batch normalization bias coefficient α, only one low-voltage amplifier unit is selected from each column of 20 low-voltage amplifier units (i.e., a total of 64 low-voltage amplifier units are selected) for correct amplification. The selected low-voltage amplifier unit is configured with a corresponding body bias voltage, and the output voltage is close to 1 / 2 VDD to ensure that each low-voltage amplifier unit achieves maximum voltage gain.

[0079] like Figure 7 As shown in Figure 1, the bias coefficient α and the accumulated bit line ABL voltage are sampled and held by a switched capacitor circuit before the main comparison stage, which can avoid the differential input offset and device mismatch of the amplifier. In order to accurately drive the input and body bias of the low-voltage amplifier unit, as shown in Figure 1, Figure 8As shown, the DAC unit in the present invention adopts a 5-bit auxiliary DAC unit based on a capacitor array and a 6-bit high-precision DAC unit, wherein the 5-bit auxiliary DAC unit pre-charges the parasitic capacitance of the input node and the body bias node of the low-voltage amplifier unit to generate a roughly estimated analog voltage, and the 6-bit high-precision DAC unit generates an accurate analog voltage based on this.

[0080] The voltage amplifier array achieves amplification based on the following steps:

[0081] 1) Select a low-voltage amplifier unit with the largest gain from the 20 low-voltage amplifier units in each column according to the bias coefficient α;

[0082] 2) The bias coefficient α is converted into an analog voltage value V1 through the aforementioned DAC unit and input to the input of the low-voltage amplifier unit selected in the previous step. The output voltage V1' of the low-voltage amplifier unit is sampled and held using the first sampling capacitor;

[0083] 3) The analog voltage value V2 on the accumulated bit line ABL is sent to the input terminal of the same low-voltage amplifier unit in step 2), and the output voltage V2' of the low-voltage amplifier unit is sampled and held using a second sampling capacitor;

[0084] 4) Since V2'-V1'>>V2-V1, the voltages output by the first sampling capacitor and the second sampling capacitor are passed through the low-voltage comparator to obtain a correct comparison result, that is, a batch normalization result is obtained.

[0085] Before implementing voltage amplification, an on-chip post-silicon tuning process is required to adjust the body bias of all low-voltage amplification units to achieve the optimal distribution characteristics. The present invention determines the optimal body bias voltage of all selected low-voltage amplification units through a body bias voltage calculation circuit, including the following steps:

[0086] a) For each column of low-voltage amplifier units, 20 pre-sized low-voltage amplifier units are input with equally spaced scan voltages ranging from 1 / 21VDD to 20 / 21VDD one by one;

[0087] b) The outputs of 20 low-voltage amplifier units with input values ​​ranging from 1 / 21VDD to 20 / 21VDD are continuously compared with a reference voltage of 1 / 2VDD via an operational amplifier (OPA). The output of the op amp (OPA) uses a closed-loop method to body-bias all NMOS transistors in the current low-voltage amplifier unit. The output body-bias voltage is further digitized by a 6-bit SAR ADC unit to obtain a digital body-bias code for the current low-voltage amplifier unit. This digital body-bias code is then stored in a 6-bit register corresponding to each low-voltage amplifier unit.

[0088] c) After selecting a low-voltage amplifier unit with the largest gain from the 20 low-voltage amplifier units in each column based on the bias coefficient α, the body bias digital code corresponding to the selected low-voltage amplifier unit is retrieved from the register, restored to an analog value through the aforementioned DAC unit, and input into the body bias node of the NMOS transistor of the corresponding low-voltage amplifier unit, thereby starting voltage amplification of the bias coefficient α and the accumulation bit line ABL.

[0089] After the tuning procedure, if Figure 9 As shown, the gain of the low voltage amplifying unit in the present invention is about 9 to 17 times, and the voltage difference between the bias coefficient α and the accumulation bit line ABL can be distinguished, and the distinguishable voltage difference is as low as 2.1mV.

[0090] Figure 10 The lazy accumulation bit line reset scheme adopted by the present invention is also illustrated, which allows the next binary accumulation cycle to be executed without resetting the residual charge on the accumulation bit line ABL. The lazy accumulation bit line reset scheme proposed by the present invention brings about a large amount of energy savings at the expense of minimal calculation errors and loss of inference accuracy. When performing binary batch normalization operations, the charge distribution between the accumulation bit line ABL and the parasitic capacitance of the low-voltage amplifier unit input node is the main cause of error, so it is necessary to properly control the reset frequency. Figure 12 As shown, when the power supply range is 0.3V to 0.4V, for VGG-16 convolution kernel sizes of 3×3×64-512 (CONV), 4096×1 (FC), and 1000×1 (FC), the present invention sets a set of reset intervals N, N=1-4, N=3, and N=1, resulting in a 0.4% to 0.3% loss in inference accuracy on the CIFAR-10 dataset. When the power supply range is 0.4V to 0.6V, for convolution kernels of different sizes, the present invention configures another set of reset intervals N, N=2-5, N=4, and N=2, resulting in a corresponding 0.3% to 0.2% loss in inference accuracy. For the two reset intervals N, the lazy accumulation bitline reset scheme provided by the present invention further reduces bitline energy by 32% and 38.7%.

[0091] The lazy accumulation bit line reset operation is illustrated as follows:

[0092] (1) When N=1: Accumulated bit line convolution -> Bias coefficient α amplification -> Accumulated bit line amplification -> Accumulated bit line reset -> Accumulated bit line convolution -> Bias coefficient α amplification -> Accumulated bit line amplification -> Accumulated bit line reset...

[0093] (2) When N=2: Accumulated bit line convolution -> Bias coefficient α amplification -> Accumulated bit line amplification -> Accumulated bit line convolution -> Bias coefficient α amplification -> Accumulated bit line amplification -> Accumulated bit line reset...

[0094] (3) When N=3: Accumulated bit line convolution -> Bias coefficient α amplification -> Accumulated bit line amplification -> Accumulated bit line convolution -> Bias coefficient α amplification -> Accumulated bit line amplification -> Accumulated bit line convolution -> Bias coefficient α amplification -> Accumulated bit line amplification -> Accumulated bit line reset...

[0095] The above 0.3V high-efficiency binary neural network accelerator requires the following steps to implement a complete convolution and batch normalization operation:

[0096] Step 1: The write channels of the bit lines BL and BLB are opened row by row through the signal on the word line WL, and the pre-trained weight values ​​W and WB are written into the 6T storage bit cells of each 10T1C multiplication bit cell through the bit lines BL and BLB respectively.

[0097] After all rows of the multiplication bit array are written with weight values, the operation of step 2 begins.

[0098] Step 2: Obtain the body bias voltage of each low-voltage amplifier unit of the voltage amplifier array.

[0099] In the first stage, 20 pre-sized low-voltage amplifier units are input with equally spaced scanning voltages ranging from 1 / 21VDD to 20 / 21VDD one by one.

[0100] In the second stage, the output of the low-voltage amplifier unit (LVA) with input values ​​ranging from 1 / 21VDD to 20 / 21VDD is continuously compared with a reference voltage of 1 / 2VDD via an op amp (OPA). The output of the OPA is used in a closed-loop manner to body-bias all NMOS transistors in the current LVA unit. The body-bias voltage is further digitized by a 6-bit SAR ADC unit to form a body-bias digital code, which is stored in a 6-bit register corresponding to each LVA unit.

[0101] In the third stage, the optimal low-voltage amplifier unit is selected according to the bias coefficient α, and the corresponding body bias digital code stored in the 6-bit register is restored to an analog value and input to the body bias node of the selected low-voltage amplifier unit NMOS, thereby starting voltage amplification of the bias coefficient α and the accumulation bit line ABL.

[0102] Step 3: Input activation values ​​F and FB from outside the chip. W and WB perform a 1-bit multiplication with F and FB via 4T1C. The multiplication result is distributed directly onto the accumulation bit line via the charge distribution of the MOM capacitor to generate the final convolution result. At this point, a single complete convolution operation is completed.

[0103] Step 4: Input the batch normalization coefficient, i.e., the bias coefficient α, from the outside of the binary neural network accelerator, and select the low-voltage amplifier unit with the largest gain based on the bias coefficient α.

[0104] The bias coefficient α is converted into an analog voltage value V1 and applied to the input of the selected low-voltage amplifier unit. The output voltage V1' of the low-voltage amplifier unit is sampled and held using the first sampling capacitor. The analog voltage value V2 of the accumulated bit line is applied to the input of the same low-voltage amplifier unit. The output voltage V2' of the low-voltage amplifier unit is sampled and held using the second sampling capacitor. Because V2'-V1'>>V2-V1, the output values ​​of the first and second sampling capacitors are compared correctly using the low-voltage comparator. At this point, the batch normalization operation is complete.

[0105] Step 5: The optional accumulation bit line ABL is reset, and the process returns to step 3 for the next convolution. If the reset interval N = 1, the reset operation takes effect after each convolution and batch normalization operation; if the reset interval N = 2, the reset operation takes effect after every second convolution and batch normalization operation; and so on for other values ​​of N.

[0106] Figure 14 The measurement results of a 4Kb in-memory binary neural network accelerator test chip made in 55nm CMOS process are shown. The chip can operate reliably with a power supply of 0.3-0.6V and consumes 2.4-91.5uW at a frequency of 12.17-129.4MHz. Figure 14 As shown in the upper left figure. At the minimum supply voltage of 0.3V, M-BC dominates the total power consumption (64.3%) of the CIM-BNN design, as shown in Figure 14 As shown in the upper right figure, in the in-memory binary neural network accelerator design of the present invention, this part of power consumption is reduced to 32.2%. Due to the proposed energy-saving multiplication bit unit and lazy accumulation bit line reset, an overall power consumption reduction of 1.9X can be achieved. Figure 14 The lower left figure shows the inference accuracy of the test chip when implementing VGG-16 and testing on CIFAR-10. While traditional batch-normalized comparators suffer significant inference accuracy loss when voltage scaling, the test chip of our invention exhibits minimal inference accuracy loss with the proposed BDVA pre-amplification scheme. When inert accumulation bitline reset is further employed, additional energy reductions of 1.47× and 1.63× are achieved at 0.3V and 0.6V, respectively, with inference accuracy losses of 0.4% and 0.2%, respectively. Figure 14 The lower middle and lower right figures show the relationship between computing density and energy efficiency indicators and VDD (0.3-0.6v), which are 0.74-7.52TOPS / mm2 and 6.06-1.62POPS / W respectively.

[0107] Figure 15This study summarizes the comparison with existing technologies. This paper implements a 4Kb in-memory binary neural network accelerator in a 55nm CMOS process, with a scalable power supply voltage down to 0.3V. The accelerator implements a binarized VGG-16 model, achieving 89.1% accuracy on CIFAR-10. The in-memory binary neural network accelerator achieves peak energy efficiency of 18.5 POPS / W and 6.06 POPS / W, representing improvements of approximately 21× and 135× over previous macromodule and system studies [9,11], respectively.

Claims

1. A high-energy-efficiency binary neural network accelerator that can be used for artificial intelligence Internet of Things, characterized by: include: A multiplication bit array, which consists of L×L sub- / near-threshold 10T1C multiplication bit cells based on series capacitors; the sub- / near-threshold 10T1C multiplication bit cell is used to perform binary multiplication and accumulation operations within one clock cycle, and consists of a 6T storage bit cell and a 4T1C memory XNOR logic cell, wherein: the 6T storage bit cell is used to store weight values ​​W and WB, W is the weight value of the pre-trained neural network, and WB is the negation of W; the 4T1C memory XNOR logic cell implements the XNOR logic between the input activation values ​​F and FB and the weights W and WB stored in the 6T storage bit cell, the capacitor in the 4T1C memory XNOR logic cell is connected in series to the accumulation bit line ABL, and the multiplication result is directly generated on the accumulation bit line ABL through charge distribution of the capacitor of the 4T1C memory XNOR logic cell to generate the final convolution result, wherein F is the binary value of the input activation value and FB is the negation of F; When writing the weight values ​​W and WB into the 6T storage bit cell, the write channels of the bit lines BL and BLB are opened row by row through the signal on the word line WL, and the pre-trained weight values ​​W and WB are written into the 6T storage bit cell of the 10T1C multiplication bit cell through the bit lines BL and BLB respectively; after the activation values ​​F and FB are input from outside the chip, the weight values ​​W and WB stored in the 6T storage bit cell are multiplied by 1 bit with the activation values ​​F and FB through the 4T1C memory XNOR logic unit, and the multiplication result is directly generated on the accumulation bit line ABL through the charge distribution of the MOM capacitor of the 4T1C memory XNOR logic unit to generate the final convolution result; A 20×L voltage amplifier array is designed with 20 low-voltage amplifier units of different transistor sizes in each column. Based on the pre-trained binary batch normalization bias coefficient α, only one low-voltage amplifier unit is selected from the 20 low-voltage amplifier units in each column for correct amplification. The selected low-voltage amplifier unit is configured with a corresponding body bias voltage, and the output voltage is close to 1 / 2VDD to ensure that each low-voltage amplifier unit obtains maximum voltage gain.

2. A high-energy-efficiency binary neural network accelerator that can be used in artificial intelligence Internet of Things according to claim 1, characterized in that: The sub- / near-threshold 10T1C multiplication bit cell adopts a multi-VT design strategy to overcome the PMOS / NMOS skew at low voltage.

3. A high-energy-efficiency binary neural network accelerator that can be used for artificial intelligence Internet of Things according to claim 2, characterized in that: The PMOS transistor used in the 6T storage bit cell is LVT, and the NMOS transistor is HVT.

4. The high-energy-efficiency binary neural network accelerator that can be used for artificial intelligence Internet of Things according to claim 1, characterized in that: All transistors of the sub- / near-threshold 10T1C multiplication bit cell adopt gate length biasing technology to reduce the influence of device mismatch, and adopt adaptive body biasing technology to reduce device deviation.

5. The high-energy-efficiency binary neural network accelerator that can be used for artificial intelligence Internet of Things according to claim 1, characterized in that: The capacitor of the 4T1C memory XNOR logic unit is implemented by a MOM capacitor with a high-level metal insert.

6. The high-energy-efficiency binary neural network accelerator applicable to artificial intelligence Internet of Things according to claim 1, characterized in that: The voltage amplification array achieves amplification based on the following steps: 1) Select a low-voltage amplifier unit with the largest gain from the 20 low-voltage amplifier units in each column according to the bias coefficient α; 2) The bias coefficient α is converted into an analog voltage value V1 through the DAC unit and input to the input of the low-voltage amplifier unit selected in the previous step. The output voltage V1' of the low-voltage amplifier unit is sampled and held using the first sampling capacitor; 3) The analog voltage value V2 on the accumulated bit line ABL is sent to the input terminal of the same low-voltage amplifier unit in step 2), and the output voltage V2' of the low-voltage amplifier unit is sampled and held using a second sampling capacitor; 4) Since V2'-V1'>>V2-V1, the voltages output by the first sampling capacitor and the second sampling capacitor are passed through the low-voltage comparator to obtain a correct comparison result, that is, a batch normalization result is obtained.

7. A high-energy-efficiency binary neural network accelerator applicable to artificial intelligence Internet of Things according to claim 6, characterized in that: Determining the optimal body bias voltages of all selected low-voltage amplifier units by a body bias voltage calculation circuit includes the following steps: a) For each column of low-voltage amplifier units, 20 pre-sized low-voltage amplifier units are input with equally spaced scan voltages ranging from 1 / 21VDD to 20 / 21VDD one by one; b) The outputs of 20 low-voltage amplifier units with input values ​​ranging from 1 / 21VDD to 20 / 21VDD are continuously compared with a reference voltage of 1 / 2VDD via an operational amplifier (OPA). The output of the op amp (OPA) uses a closed-loop method to body-bias all NMOS transistors in the current low-voltage amplifier unit. The output body-bias voltage is further digitized by a 6-bit SAR ADC unit to obtain a digital body-bias code for the current low-voltage amplifier unit. This digital body-bias code is then stored in a 6-bit register corresponding to each low-voltage amplifier unit. c) After selecting a low-voltage amplifier unit with the largest gain from the 20 low-voltage amplifier units in each column based on the bias coefficient α, the body bias digital code corresponding to the selected low-voltage amplifier unit is retrieved from the register, restored to an analog value through the DAC unit, and input into the body bias node of the NMOS transistor of the corresponding low-voltage amplifier unit, thereby starting voltage amplification of the bias coefficient α and the accumulation bit line ABL.

8. The high-energy-efficiency binary neural network accelerator applicable to artificial intelligence Internet of Things according to claim 7, characterized in that: The DAC unit adopts a 5-bit auxiliary DAC unit based on a capacitor array and a 6-bit high-precision DAC unit. The 5-bit auxiliary DAC unit pre-charges the parasitic capacitance of the input node and body bias node of the low-voltage amplifier unit to generate a roughly estimated analog voltage, and the 6-bit high-precision DAC unit generates an accurate analog voltage based on this.

9. The high-energy-efficiency binary neural network accelerator applicable to artificial intelligence Internet of Things according to claim 7, characterized in that: The accumulation bit line ABL adopts an inert accumulation bit line reset scheme, which sets a reset interval N according to different power supply voltages and the convolution kernel size adopted by the neural network. The reset operation of the accumulation bit line ABL takes effect after the Nth convolution and batch normalization operation.

Images