A low temperature high energy efficient in-memory computing accelerator

Abstract

The application discloses a low-temperature high-energy-efficiency in-memory computing accelerator.The application has the following innovations: a high-retention-time low-temperature 3T memory cell design: the application proposes a low-temperature 3T memory cell design based on eDRAM, which can significantly improve the retention time without any word line voltage boosting scheme, and realize full-swing data transmission during the write operation process; a low-temperature adaptive reconfigurable sensitive amplifier design: the application develops a low-temperature on-chip adaptive reconfigurable sensitive amplifier design, which can realize accurate on-chip Boolean logic calculation by configuring the reference voltage of ARSA; a low-temperature optimized Flash ADC design: the application uses the designed ARSA to adaptively generate 15 ARSA reference voltages on the chip and reconfigure them into 4bit Flash ADCs.Through the adaptive configuration of the reference voltage and the storage mode on the chip, the design can ensure the realization of fast and low-power consumption convolution calculation.

Description

TECHNICAL FIELD

[0001] The present application relates to a design of a low-temperature high-energy-efficient in-memory computing accelerator (CIMC). BACKGROUND

[0002] As the integrated circuit industry reaches a bottleneck following Moore's law, more and more research is being done to find alternative technologies and architectures to further improve performance. The characteristics of CMOS approaching ideal performance in a low-temperature environment [1][2] further promote the development of low-temperature applications, and low-temperature computing has also received considerable attention in the past few years. However, low-temperature computing cannot eliminate the current performance bottleneck, such as the memory wall. In order to solve the above problems, low-temperature computing architecture based on in-memory computing is a very promising solution. They are suitable for running at low temperature, reduce cooling costs by extremely high energy efficiency, and achieve high-energy-efficient computing and storage capabilities with relatively small adjustments to the architecture.

[0003] However, existing in-memory computing research [3-7] still has several challenges in improving energy efficiency at low temperature: existing low-temperature eDRAM is not optimal in terms of implementing reliable write operations, and its memory cell topology needs to be redesigned at low temperature; the demand for different computing operations in different low-temperature computing scenarios, the need for high-energy-efficient Boolean logic computing implementation, and high-energy-efficient convolution operations.

[0004] REFERENCES:

[0005] [1] D. Min, I. Byun, G.-H. Lee, S. Na, and J. Kim, “Cryocache: A fast, large, and cost-effective cache architecture for cryogenic computing,” in Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS’20. New York, NY, USA: Association for Computing Machinery, Mar. 2020, p. 449-464.

[0006] [2] I. Byun, D. Min, G.-h. Lee, S. Na, and J. Kim, “Cryocore: A fast and dense processor architecture for cryogenic computing,” in 2020 ACM / IEEE 47th Annual International Symposium on Computer Architecture (ISCA), May 2020, pp. 335-348.

[0007] [3] Chen, Zhengyu, Xi Chen, and Jie Gu. “15.3A 65nm 3T Dynamic Analog RAM-Based Computing-in-Memory Macro and CNN Accelerator with Retention Enhancement, Adaptive Analog Sparsity and 44TOPS / W System Energy Efficiency.” 2021 IEEE International Solid-State Circuits Conference (ISSCC). Vol. 64. IEEE, 2021.

[0008] [4] Xie, Shanshan, et al. “16.2 eDRAM-CIM: compute-in-memory design with reconfigurable embedded-dynamic-memory array realizing adaptive data converters and charge-domain computing.” 2021 IEEE International Solid-State Circuits Conference (ISSCC). Vol. 64. IEEE, 2021.

[0009] [5] Dong, Qing, et al. "15.3A 351TOPS / W and 372.4GOPS compute-in-memory SRAM macro in 7nm FinFET CMOS for machine-learning applications." 2020 IEEE International Solid-State Circuits Conference-(ISSCC). IEEE, 2020.

[0010] [6] Fujiwara, Hidehiro, et al. "A 5-nm 254-TOPS / W 221-TOPS / mm2 Fully-Digital Computing-in-Memory Macro Supporting Wide-Range Dynamic-Voltage-Frequency Scaling and Simultaneous MAC and Write Operations." 2022 IEEE International Solid-State Circuits Conference (ISSCC). Vol. 65. IEEE, 2022.

[0011] [7] Si, Xin, et al. "24.5A twin-8T SRAM computation-in-memory macro for multiple-bit CNN-based machine learning." 2019 IEEE International Solid-State Circuits Conference-(ISSCC). IEEE, 2019. SUMMARY

[0012] The technical problem to be solved by the present application is that the existing low-temperature eDRAM is not optimal in terms of implementing reliable write operations, and the storage cell topology needs to be redesigned at low temperature; the demand for different computing operations in different low-temperature computing scenarios requires high-energy-efficiency Boolean logic computing implementation and high-energy-efficiency convolution operation.

[0013] In order to solve the above technical problems, the technical scheme of the present application is to provide a low-temperature high-energy-efficiency in-memory computing accelerator, characterized in that it comprises a C3T macro, each C3T macro comprises a memory cell C3T array of M rows by N columns, an input signal is converted into a timing signal with a corresponding pulse width by a digital timing converter array and controls the charging and discharging of the memory cell C3T in the corresponding row of the C3T macro to the bit line RBL of the corresponding column; the voltage on the corresponding column bit line RBL is sampled by a sensitive amplifier arranged in each C3T macro to obtain the final result, wherein:

[0014] In the non-convolution operation mode, the corresponding column bit line RBL is directly connected to the sensitive amplifier;

[0015] In the convolution operation mode, by controlling the on-off of the switch: first, connect the same size of convolution capacitor to each column bit line RBL; after completing the charging and discharging of the convolution capacitor, connect the adjacent two column bit lines RBL together to realize the charge redistribution between different columns; finally, disconnect the connection between the bit line RBL and the sensitive amplifier, and make the different sizes of charge on different columns be sampled by the sensitive amplifier and generate the final output result.

[0016] Preferably, the memory cell C3T comprises a pair of complementary CMOS structure transmission gate write port and a read port composed of a single tube NMOS; for the write operation, the stored data is written to the storage node SN through the write bit line WBL and the transmission gate write port controlled by a pair of write word lines WWL, WWLB; for the read operation, different charging and discharging behaviors of the bit line RBL are completed by controlling the pulse width length of the read signal RWL.

[0017] Preferably, a transmission gate switch and a storage capacitor are arranged at the two input ends of the sensitive amplifier respectively, and the sampling transistor at each side of the input end of the sensitive amplifier and the transmission gate switch constitute a storage node for storing the sampling voltage V REF ; during the sampling process, the voltage on the bit line RBL is locked in V REF through the transmission gate switch at one side of the sensitive amplifier; after the sampling voltage is locked, the transmission gate switch at one side of the sensitive amplifier is in the off state to ensure that the sampling voltage is not affected by the voltage change on the bit line RBL and is always stored in V REF , while the actual calculation result is sampled through the transmission gate switch at the other side of the sensitive amplifier and compared with the stored V REF to generate the final output result.

[0018] Preferably, the implementation of Boolean calculation comprises the following steps:

[0019] Store the reference data of the corresponding sampling voltage in the C3T macro;

[0020] opening the word lines of the C3T macro multiple rows to generate corresponding column-wise results;

[0021] connecting between adjacent column bit lines RBL to obtain charge redistribution results;

[0022] storing the charge redistribution results to the corresponding column's sense amplifiers and latching in V REF , where for any input NAND or NOR operation, the reference voltage for the decision result is generated and stored into the sense amplifiers to realize the corresponding calculation operation.

[0023] Preferably, a single 4-bit Flash ADC is composed of 15 sense amplifiers in the C3T macro, and 15 adaptive V REF are generated before the convolution operation.

[0024] Compared with the prior art, the innovation of the present application is:

[0025] 1) High retention time low temperature 3T memory cell (C3T) design: The present application proposes a low temperature 3T memory cell design based on eDRAM, which can significantly improve the retention time without any word line voltage boosting scheme, and realizes full swing data transmission during the write operation.

[0026] 2) Low temperature adaptive reconfigurable sense amplifier design (ARSA): The present application develops a low temperature on-chip adaptive reconfigurable sense amplifier design, which can realize accurate on-chip Boolean logic calculation by configuring the reference voltage of ARSA.

[0027] 3) Low temperature optimized Flash ADC design: The present application uses the designed ARSA to adaptively generate 15 reference voltages of ARSA on-chip, and reconfigure them into a 4-bit Flash ADC. Through the adaptive configuration of the reference voltage and the storage mode on-chip, this design can ensure fast and low power consumption convolution calculation implementation.

[0028] The chip test results show that compared with the 3.7us data retention time at 300K, the retention time of the C3T design disclosed in the present application is improved to 9.1s at 4.2K. The 144Kb CIMC of the present application realizes an average energy efficiency of 603.1TOPS / W and an average calculation density of 284TOPS / mm 2 , which are 2.37 times and 1.29 times higher than the most advanced 5nm technology research work [6], respectively. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 Low temperature in-memory computing architecture design diagram (C3T array, ARSA and low temperature Flash ADC);

[0030] Figure 2 The design of the C3T memory unit and the control signals for different operating modes are illustrated.

[0031] Figure 3 This illustrates an adaptive reconfigurable sensitive amplifier (ARSA) design.

[0032] Figure 4 This diagram illustrates the implementation of Boolean logic based on ARSA.

[0033] Figure 5 This diagram illustrates an ARSA-based Flash ADC design: Adaptive V REF Generation, convolution process, and measurement results;

[0034] Figure 6 The results of CIMC's retention time, accuracy, energy efficiency, and power consumption measurements are illustrated.

[0035] Figure 7 This illustration summarizes the design of the present invention and compares it with state-of-the-art research. Detailed Implementation

[0036] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0037] like Figure 1 As shown, the 144Kb CIMC architecture disclosed in this embodiment includes a digital timing converter (DTC) array, 64 C3T tiles, an ARSA array, ReLU, a read / write interface (R / W interface), and other peripheral circuitry supporting conventional memory operations. The input signal is converted into a timing signal of the corresponding pulse width by the DTC array and controls the charging and discharging of the corresponding row of memory cells C3T on the bit line RBL. The voltage on the bit line RBL is sampled by a sensitive amplifier configured in each C3T tile to obtain the final result. During non-convolution operations, to save charging energy from the large load capacitors on the bit line RBL, this invention disconnects the convolutional capacitors from the bit line RBL, that is... Figure 1SW3-SW6 in the lower right corner of the figure will be in the off state, while the switch SW7 is in the closed state to achieve the connection of the bit line RBL and the sensitive amplifier. In the convolution operation mode, by closing the switches SW5-SW7, the 8C0 size of the convolution capacitor is connected to each column of bit line RBL. After completing the charge and discharge of the convolution capacitor, SW3-SW4 is closed to achieve the charge redistribution between different columns. Finally, the switch SW7 is disconnected, at this time, only the charges on 8C0, 4C0, 2C0 and C0 of different columns will be sampled by the sensitive amplifier and the final output result will be generated.

[0038] In combination Figure 2 Although the single type write access tube (N type or P type) used in the normal temperature eDRAM design can effectively reduce the leakage of data at the storage node SN, the full swing data write problem caused by threshold voltage drop cannot be avoided. This situation is more serious at low temperature. The power consumption and device life impact generated by the word line voltage boosting solution at low temperature also makes this structure not suitable for low temperature design. In addition, the charge injection effect from the write word line WWL to the storage node SN further causes the decay of the data storage after the write operation. In order to solve this problem, the C3T gain cell design is proposed, which includes a pair of write ports composed of transmission gates (P1 and N1), and a read port composed of a single tube NMOS (N2). The storage data is written into the storage node SN in the storage cell through the write bit line WBL and the transmission gate write port controlled by a pair of write word lines WWL, WWLB. For the read operation, according to the design of the application, the storage cell supports Boolean operation and convolution operation in addition to the conventional storage operation, which is mainly realized by controlling the pulse width length of the read signal RWL to complete the different charge and discharge behaviors of the read word line RBL. As Figure 2 As shown in the timing diagram in the lower left corner, due to the use of a pair of complementary CMOS structure composed of transmission gate write port, any storage data can be stored in the storage node SN through the structure, and the structure can also eliminate the influence of charge injection effect on the storage data.

[0039] As Figure 3 As shown in the timing diagram in the lower left corner, due to the use of a pair of complementary CMOS structure composed of transmission gate write port, any storage data can be stored in the storage node SN through the structure, and the structure can also eliminate the influence of charge injection effect on the storage data. REF. Because the structure of storing the sampling voltage is similar to the C3T cell designed in this application, it is called C3T-like. The complete operation of ARSA is as follows: first, in the sampling process, the voltage on the bit line RBL is latched in V REF after passing through the switch SW1 composed of S1 / S1B. After the latching of the sampling voltage is completed, SW1 will be in the off state to ensure that the sampling voltage is not affected by the voltage on the bit line RBL and is stored in V REF at all times, while the actual calculation result will be sampled through the switch SW2 composed of S2 / S2B and compared with the stored V REF to generate the final output result.

[0040] As shown in Figure 4 , to realize Boolean calculation, first, the reference data (REF Data) of the corresponding sampling voltage needs to be stored in the storage array, and then the word lines of multiple rows are opened to generate the corresponding column direction result. Next, the connection between adjacent columns through the column switch SW3 is needed to obtain the charge redistribution result. Finally, the result is stored in the ARSA of the corresponding column and latched in V REF . For any input NAND or NOR operation, only the reference voltage for judging the result needs to be generated according to the above process and stored in the ARSA to realize the corresponding calculation operation. After the storage of the reference data is completed, the gating of multiple rows is controlled through the read signal RWL and the result is generated on the column. Then, adjacent columns are connected together through the column switch SW3 and share the result. After that, the result is stored in the ARSA to obtain the first reference voltage V REF [1]. To generate V REF [2] or other reference voltages, only the corresponding row needs to be gated and the above operation needs to be repeated.

[0041] Figure 5 The upper left shows the structure of reconstructing 15V REF for a 4-bit Flash ADC, which also shows the charge redistribution process of 4-bit convolution operation. A single 4-bit Flash ADC is composed of 15 ARSAs in the C3T Tile, and 15V REF is generated before the convolution operation. Figure 5 The upper right shows the pre-sampling process of adaptive 15V REF . In the first cycle (cycle 1), RBL[1:4] will be discharged to different voltage levels according to the number of "1"s stored in each column. The C3T array is divided into 30 parts, each containing 19 rows (the array size is 576 rows x 256 columns, 576 rows / 30≈19 rows). For example, to obtain V REF [1] and V REF[2] We store 19×1 '1's into the first column of the C3T Tile and write 19×3 '1's into the second column. In this case, the voltages of RBL[1] and RBL[2] will be respectively (V H -V L ) / 30 and 3(V H -V L The voltage drop decreases by 30% (V) H and V L (Refers to the maximum and minimum values ​​calculated by convolution).

[0042] exist Figure 5 The lower left corner shows the CIMC convolution operation flow and corresponding data mapping rules. The input activation value (IA) generates a corresponding time pulse signal via DTC. After all rows are opened, convolution calculations can be performed through charge sharing, generating a voltage V on the bit line RBL. RBL By using V RBL With presampling V REF By making comparisons, we can obtain the final result. Figure 5 The measurement results of the 4-bit Flash ADC are shown in the lower right corner. The linearity of the convolution calculation was verified by changing the number of stored '1's in the column. The results show that the structure has a good linear ADC output. Compared with the resistive trapezoidal ADC design, the 4-bit Flash ADC composed of ARSAs reduces area and power consumption by 2.6 times and 23.8 times, respectively, at a temperature of 4.2K.

[0043] Figure 6 The results presented are measurement results for a 144Kb C3T macrochip fabricated using a 40nm process. For retention time (RT), a 0.1V data voltage change was used as the critical condition to trigger a data refresh operation. Compared to the 3.7µs RT at 300K, the C3T macro of this invention (i.e., the "C3T Tile") has an average RT of 9.1s at 4.2K. For Boolean calculations, this C3T macro can achieve accurate calculations over a long period without refreshing the ARSA reference voltage. For convolution calculations, this invention achieves an energy efficiency of 603.1 TOPS / W, which is 6.52 times that of the 300K test results. Furthermore, this invention also achieves up to 284 TOPS / mm². 2The power breakdown of the chip shows that the power consumption overhead of the Flash ADC is as high as 86.17% at 300K temperature, while the present application can reduce it to 23.62% at 4.2K. For the ResNet-18 model, the C3T macro at 4.2K realizes the highest 93.17% accuracy of CIFAR-10 inference. The maximum accuracy loss is 0.05% within the retention time. In addition, the work maintains the CIFAR-100 accuracy of 68.23%-68.12% at 4.2K, with a maximum accuracy loss of 0.11%.

[0044] As shown in Figure 7 , the present application realizes a macro module design of up to 144Kb in a 40nm CMOS process, while maintaining high computing density and improving computing energy efficiency. The CIMC realizes an energy efficiency of 603TOPS / W, which is 2.37 times higher than the most advanced 5nm technology research [6]. This work can also realize a computing density of 284TOPS / mm 2 .

Claims

1. A low temperature, high energy efficient in-memory computing accelerator, comprising: The C3T macro includes an M-row-by-N-column memory cell C3T array, an input signal is converted into a corresponding pulse width timing signal by a digital timing converter array and controls the charging and discharging of a corresponding column bit line RBL by the corresponding row memory cell C3T in the C3T macro; and a final result is obtained by sampling the voltage on the corresponding column bit line RBL by a sense amplifier arranged in each C3T macro, wherein: in a non-convolution operation mode, the corresponding column bit line RBL is directly connected to the sense amplifier; in a convolution operation mode, by controlling the on-off of a switch: a same size convolution capacitor is first connected to each column bit line RBL; after the charging and discharging of the convolution capacitor is completed, the adjacent two column bit lines RBL are connected together to achieve charge redistribution between different columns; finally, the connection between the bit line RBL and the sense amplifier is disconnected, and different sizes of charges on different columns are sampled by the sense amplifier to generate the final output result; the memory cell C3T includes a pair of complementary CMOS structure transmission gate write ports and a read port composed of a single tube NMOS; for the write operation, the stored data is written to the storage node SN through the write bit line WBL and the transmission gate write port controlled by a pair of write word lines WWL and WWLB; for the read operation, different charging and discharging behaviors of the bit line RBL are completed by controlling the pulse width length of the read signal RWL.

2. The cryogenic in-memory computing accelerator of claim 1, wherein, A transmission gate switch and a storage capacitor are set at each input terminal of the sense amplifier, and the sampling transistor and the transmission gate switch at each input terminal of the sense amplifier form a storage node for storing the sampling voltage V REF During the sampling process, the voltage on the bit line RBL is locked in V REF by the transmission gate switch at one side of the sense amplifier; after the sampling voltage is locked, the transmission gate switch at one side of the sense amplifier is turned off to ensure that the sampling voltage is not affected by the voltage on the bit line RBL and is stored in V REF , while the actual calculation result is sampled by the transmission gate switch at the other side of the sense amplifier and compared with the stored V REF to generate the final output result.

3. The cryogenic in-memory computing accelerator of claim 2, wherein, Implementing Boolean computation includes the following steps: storing reference data of corresponding sampling voltage into the C3T macro; opening the word lines of multiple rows of the C3T macro to generate corresponding column direction results; connecting between adjacent column bit lines RBL to obtain charge redistribution results; storing the charge redistribution results to the corresponding column of the sense amplifier and locking in V REF Wherein, for any input NAND or NOR operation, the reference voltage for judging the result is generated and stored into the sense amplifier, and the corresponding calculation operation can be realized.

4. The cryogenic in-memory computing accelerator of claim 3, wherein, A single 4-bit Flash ADC is composed of 15 sensitive amplifiers in C3T macro and produces adaptive 15 V REF .

Images