MIXED SRAM AND eDRAM CELL FOR AREA AND ENERGY-EFFICIENT ON-CHIP AI MEMORY

Abstract

The disclosed technology herein relates to an on-chip artificial intelligence memory system. The system includes a mixed-memory cell comprising a static random access memory (SRAM) cell and at least one embedded dynamic random access memory (eDRAM) cell. The mixed-memory cell is configured to map a control bit of encoded data to the SRAM cell and to map remaining bits of the encoded data to the at least one eDRAM cell. The system includes an enhancement encoder decoder module configured to receive and encode data for storage in the mixed-memory cell. The system includes a refresh controller configured to refresh at least one data bit stored in the at least one eDRAM cell.

Description

Docket # 047162-5382-OOWOMIXED SRAM AND eDRAM CELL FOR AREA AND ENERGY-EFFICIENT ON-CHIP Al MEMORYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to US Provisional Patent Application No. 63 / 723,362, filed on November 21, 2024, incorporated herein by reference in its entirety.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] This invention was made with government support under Grant Nos. 2312366 and 2318152 awarded by the National Science Foundation and Grant No. DE-SC0023198 awarded by the U.S. Department of Energy. The government has certain rights in the invention.BACKGROUND

[0003] Deep Neural Network (DNN) accelerators have become crucial components in various machine learning systems. DNNs store a large number of parameters to achieve high accuracy, resulting in high memory requirements. DNNs have proven their effectiveness in a wide range of applications, including image recognition (Krizhevsky, et al., NIPS, 2012), object detection (Huang, et al., IEEE, 2018), language translation (Floridi, et al., Minds and Machines, 2020), and autonomous driving (Yurtsever, et al., IEEE Access, 2020). State-of-the-art DNNs require billions of operations and a huge memory to store activations and weights, as evidenced by the 240* increase in transformer size over two years (Gholami, et al., RiseLab Medium Post, 2021). Dedicated memory hierarchies have been designed to balance the low-cost storage provided by off-chip dynamic random-access memory (DRAMs) and the energy-efficient access offered by on-chip static random-access memory (SRAMs) (Chen, et al., JSSCC, 2016). This trend has led to an increase in the use of larger on-chip memory in cutting-edge DNN accelerators. For instance, SRAM accounts for 79.2% of the chip area and 42.5% of the power consumption in Eyeriss, as illustrated in FIG. 1 A, 67% of the chip area in chiplet designs like Simba (Shao, et al., IEEE, 2019), and the latest wafer-scale chips house up to 18 GB of on-chip memory (Celebras, 2020). Thus, the use of on-chip SRAM memories results in higher power & area requirements.Docket # 047162-5382-OOWO

[0004] 6T (six-transistor) SRAMs have long been the preferred embedded memory choice because of their logic-compatible bit-cell, quick differential read, and static data retention (Khan, et al., 5th International Conference on Design & Technology of Integrated Systems in Nanoscale Era, 2010). However, their relatively large cell size and competing requirements for reading and writing at low operating voltages make scaling 6T SRAMs difficult in advanced Complementary Metal-Oxide-Semiconductor (CMOS) technologies (Khan, et al., 5th International Conference on Design & Technology of Integrated Systems in Nanoscale Era, 2010). Recently, nonvolatile memories have captured the research community’s interest due to their small cell size, low cell leakage, and fast read access operation. Earlier studies have attempted to replace on-chip SRAM with nonvolatile memories like ReRAM, FeFET, and others (see Long, et al., 2019 Design, Automation & Test in Europe Conference & Exhibition, 2019; Reis, et al., Proceedings of the international symposium on low power electronics and design, 2018). Nonetheless, the write operation in a nonvolatile memory is slower and consumes higher energy than the read operation (see Inci, et al., IEEE, 2021; Mittal, et al., IEEE, 2015; and Yu, et al., IEEE circuits and systems magazine, 2021). This can negatively impact the performance of Artificial Intelligence (Al) chips in DNN applications, such as onchip learning, where both on-chip read and write operations are imperative (see Peng, et al., IEEE, 2020). Another alternative to on-chip SRAM is embedded dynamic random-access memory (eDRAM). Table 1 below presents comparisons across different embedded memories on the same 65 nm low power CMOS process (see Chun, et al., IEEE Journal of Solid-State Circuits, 2011).

[0005] As disclosed herein, 1T1C eDRAM (1 transistor and 1 capacitor) offers 4.5x higher bitcell density and 5. Ox lower static power dissipation than 6T SRAMs, even including refresh power. This results in a smaller chip size, faster memory access and increased memory density, which are the most effective methods to enhance the microprocessor performance within givenDocket # 047162-5382-OOWO power constraints. However, nonvolatile memory and conventional eDRAM (1T1 C) entail a complex fabrication process, as they require specialized materials for wafer deployment (see Ali, et al., IEEE Transactions on Circuits and Systems I: Regular Papers, 2019).

[0006] 3T (three-transistor) and 2T (two-transistor) CMOS eDRAM gain cell designs are embedded dynamic random-access memory circuits that utilize fewer transistors per memory cell than traditional SRAM. This results in increased density and smaller area. 3T / 2T eDRAM cells are made using logic devices, enabling their construction in standard CMOS processes with minimal modifications. Industrial designs have shown that three transistors can be used to achieve approximately 2* higher bit-cell density than SRAM. To that end, the eDRAM gain cell (3T and 2T) can reduce the on-chip SRAM area without altering the fabrication technology (see Chun, et al., IEEE Journal of Solid-State Circuits, 2011; Chun, et al., IEEE Journal of Solid-State Circuits, 2011; and Ichihashi, et al., 2005 Symposium on VLSI Circuits, 2005). As shown in Table 1 above, the eDRAM gain cell provides both area and energy advantages compared to on- chip SRAM. Specifically, the 2T eDRAM offers a 5.26* reduction in static power dissipation compared to SRAM. However, the use of eDRAM gain cells results in significant refresh power consumption due to smaller retention times, thereby limiting the power advantages of the eDRAM gain cells with respect to on-chip SRAM. Consequently, implementing the eDRAM gain cells in Al chips remains a viable consideration.

[0007] In deep learning applications, 8-bit integers (INT8) have emerged as the ideal numerical representation, maintaining accuracy across a wide range of tasks (see Jouppi, et al., ISCA, 2021). In the 8-bit integer format, a standard for DNN quantization, errors occurring in the Most Significant Bits (MSBs) carry more weight than those in the Least Significant Bits (LSBs) (Li, et al., ACM, 2017). The 8-bit integer data from quantized DNNs tend to cluster around zero (Nguyen, et al., IEEE Access, 2021). For such small integers near zero, the MSBs are usually zeros for positive values and ones for negative values. This pattern offers a chance to increase the number of ones in positive integers by bit flipping, thus creating a dominance of ones in DNN data. The LSBs, more populated by zero bits, can bear errors with minimal effect on the final accuracy owing to their lesser significance. A recent study presented an asymmetric DNN data-encoder that boosts the frequency of bit-0 in the INT8 representations while preserving DNN performance (Nguyen, et al., IEEE Access, 2021). This idea can be further exploited inDocket # 047162-5382-OOWO conjunction with on-chip data storage using 2T eDRAM that demonstrates an asymmetry in data retention between bit-1 and bit-O, where bit-1 offers reduced static and access energy compared to bit-0 (Chun, et al., IEEE Journal of Solid-State Circuits, 2011). Integrating a hybrid 6T SRAM / 2T eDRAM design with a one-enhancement data-encoder (that enhances the prevalence of bit-1 in INT8 representations) can optimize for both area and energy consumption on-chip. In the context of hybrid SRAM and eDRAM architectures, a low-voltage design may combine 8T SRAM and 3T eDRAM specifically for video applications (Kazimirsky, et al., IEEE, 2017). The advantage of this approach is particularly evident in the context of video frames’ short duration, a period during which the human eye is generally incapable of perceiving minor discrepancies. However, the question of whether similar approximations will affect machine recognition capabilities in the same way they are imperceptible to humans remains unanswered in machine learning applications.

[0008] Thus, there is a need in the art for a mixed memory cell based on an SRAM and asymmetric eDRAM design for area and energy-efficient on-chip Al memory (MCAIMem), which is adaptable, capable of accommodating various memory capacities and performance needs.SUMMARY

[0009] The present disclosure relates generally to on-chip Al memory, and more particularly, to mixed memory cell based on SRAM and asymmetric eDRAM designed for area and energyefficient on-chip Al memory (MCAIMem). Such MCAIMem is adaptable, capable of accommodating various memory capacities and performance needs, making it appropriate for a broad spectrum of Al applications, from compact edge devices to extensive data centers.

[0010] The disclosed technology herein relates to an on-chip artificial intelligence memory system. The system includes a mixed-memory cell comprising a static random access memory (SRAM) cell and at least one embedded dynamic random access memory (eDRAM) cell. The mixed-memory cell is configured to map a control bit of encoded data to the SRAM cell and to map remaining bits of the encoded data to the at least one eDRAM cell. The system includes an enhancement encoder decoder module configured to receive and encode data for storage in theDocket # 047162-5382-OOWO mixed-memory cell. The system includes a refresh controller configured to refresh at least one data bit stored in the at least one eDRAM cell. .

[0011] A variety of additional aspects will be set forth in the description that follows. The aspects can relate to individual features or to combinations of features. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the broad inventive concepts upon which the embodiments disclosed herein are based.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The following drawings are illustrative of particular embodiments of the present disclosure and therefore do not limit the scope of the present disclosure. The drawings are not to scale and are intended for use in conjunction with the explanations in the following detailed description.

[0013] FIG. 1A illustrates a breakdown of SRAM area and power in an Eyeriss Chip. FIG. IB is a graph of experimental data showing an eDRAM gain cell maintaining functionality as retention time diminishes at scaled technology nodes.

[0014] FIG. 2A illustrates a cell design and data retention time of 3T eDRAM. FIG. 2B illustrates a cell design and data retention time of 2T eDRAM. FIG. 2C is a circuit diagram showing a current sense amplifier of 2T eDRAM.

[0015] FIG. 3 A illustrates an 8-bit integer represented in two’s complement form, showcasing the binary encoding of both positive and negative integers. FIG. 3B illustrates the impact of the One Enhancement Encoder / Decoder, in which the original bit data is converted to 1 -bit data based on its sign bit. FIG. 3C illustrates a bit histogram of weight data in a ResNet-50 deep learning network, before and after using the one-enhancement encoder.

[0016] FIG. 4A is a diagram of an example architecture of one embodiment of the disclosed MCAIMem and the one-enhancement encoder / decoder. FIG. 4B is a flowchart illustrating an example process of storing data to the MCAIMem.Docket # 047162-5382-OOWO

[0017] FIGs. 5A and 5B illustrate a bit statistic histogram comparison for weights and activations pre-function and post-function of the one-enhancement encoder / decoder.

[0018] FIG. 6A illustrates a modified design of a 2T eDRAM. FIG. 6B is a graph of retention time and NMOS width impact on 2T eDRAM. FIG. 6C is set of layout designs of the 6T SRAM, a traditional 2T eDRAM, and an enhanced 2T eDRAM cell.

[0019] FIG. 7A is a circuit diagram of a 6T SRAM cell and a 2T eDRAM cell design. FIG. 7B illustrates a connectivity between 6T SRAM and 2T eDRAM with a multiple purpose sense amplifier. FIG. 7C illustrates an amplifier circuit design.

[0020] FIG. 8A illustrates the readability and writability of 6T SRAM with various access transistor configurations. FIG. 8B is a comparison of the write yield of 6T SRAM with different access transistor configurations.

[0021] FIG. 9A is a timing diagram of a write operation in the MCAIMem. FIG. 9B is a timing diagram of a read operation in the MCAIMem.

[0022] FIGs. 10A and 10B illustrate the impact of retention errors on the accuracy of Deep Neural Networks (DNNs) when using the MCAIMem, with and without the one-enhancement technique.

[0023] FIG. 11 A illustrates a statistical development method of 0-to-l flipping error probability model of MCAIMem using a Monte Carlo simulation and REF. FIG. 1 IB illustrates a 0-to-l flipping error probability model with various VREF values.

[0024] FIG. 12 is a diagram showing a surface area comparison between a 16KB bank layout of MCAIMem and an equivalent 6T SRAM 16KB bank layout.

[0025] FIG. 13A is a graph of comparative energy consumption of SRAM, eDRAM (with / without refresh), MCAIMem (with / without refresh), and RRAM. FIG. 13B is a graph of normalized Ops / Watt for SRAM, eDRAM, MCAIMem, and RRAM across various DNN benchmarks on Eyeriss and TPUvl at 100MHz.

[0026] FIG. 14 is a graph of total energy consumption across varying clock frequencies for DNN benchmarks on Eyeriss and TPUvl.Docket # 047162-5382-OOWO

[0027] FIG. 15 is a graph comparing size (surface area) and power of a Mixed SRAM and embedded Random Access Memory (eDRAM) cell for area and energy-efficient on-chip Al memory (MCAIMem) to that of a conventional 6T SRAM.DETAILED DESCRIPTION

[0028] The following discussion omits or only briefly describes conventional features of on-chip Al memory that are apparent to those skilled in the art. It is noted that various embodiments are described in detail with reference to the drawings, in which like reference numerals represent like parts and assemblies throughout the several views. Reference to various embodiments does not limit the scope of the claims attached hereto. Additionally, any examples set forth in this specification are intended to be non-limiting and merely set forth some of the many possible embodiments for the appended claims. Further, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations.

[0029] Unless otherwise specifically defined herein, all terms are to be given their broadest reasonable interpretation including meanings implied from the specification as well as meanings understood by those skilled in the art and / or as defined in dictionaries, treatises, etc. It is noted that, as used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless otherwise specified, and that the terms “includes” and / or “including,” when used in this specification, specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0030] Terms concerning attachments, coupling and the like, such as “connected” and “interconnected,” refer to a relationship wherein structures are secured or attached to one another either directly or indirectly through intervening structures, as well as both movable or rigid attachments or relationships, unless expressly described otherwise. The term “operatively or operably connected” is such an attachment, coupling or connection that allows the pertinent structures to operate as intended by virtue of that relationship.

[0031] Reference throughout the specification to “one embodiment”, “an embodiment” or “some embodiments” means that a particular feature, structure, or characteristic described in connectionDocket # 047162-5382-OOWO with an embodiment is included in at least one embodiment of the subject matter disclosed. Thus, the appearance of the phrases “in one embodiment”, “in an embodiment” or “in some embodiments” in various places throughout the specification is not necessarily referring to the same embodiment. Further, the particular features, structures or characteristics of “one embodiment”, “an embodiment” or “some embodiments” may be combined in any suitable manner with each other to form additional embodiments of such combinations. It is intended that embodiments of the disclosed subject matter cover modifications and variations thereof. Terms such as “first,” “second,” “third,” etc., merely identify one of a number of portions, components, steps, operations, functions, and / or points of reference as disclosed herein, and likewise do not necessarily limit embodiments of the present disclosure to any particular configuration or orientation.

[0032] Moreover, throughout this disclosure, various aspects of the invention can be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the invention. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., as well as individual numbers within that range, for example, 1, 2, 2.7, 3, 4, 5, 5.3, 6, and any whole and partial increments there between. This applies regardless of the breadth of the range. As used herein, the term “about” in reference to a measurable value, such as an amount, a temporal duration, and the like, is meant to encompass the specified value or variations of plus or minus 20%, plus or minus 10%, plus or minus 5%, plus or minus 1%, and plus or minus 0.1% of the specified value, as such variations are appropriate.

[0033] Al chips commonly employ SRAM memory as buffers for their reliability and speed, which contribute to high performance. However, SRAM is expensive and demands significant area and energy consumption. Previous studies have explored replacing SRAM with emerging technologies like nonvolatile memory, which offers fast-read memory access and a small cell area. Despite these advantages, non-volatile memory’s slow write memory access and high write energy consumption prevent it from surpassing SRAM performance in Al applications withDocket # 047162-5382-OOWO extensive memory access requirements. Some research has also investigated eDRAM as an areaefficient on-chip memory with similar access times as SRAM. Still, refresh power remains a concern, leaving the trade-off between performance, area, and power consumption unresolved. To address this issue, the disclosure provided herein describes a novel mixed CMOS cell memory design that balances performance, area, and energy efficiency for Al memory by combining SRAM and eDRAM cells. The disclosure describes a proportion ratio of one SRAM and seven eDRAM cells in the memory to achieve area reduction using mixed CMOS cell memory. Additionally, the disclosure describes the characteristics of DNN data representation and integrates asymmetric eDRAM cells to lower energy consumption. To validate the MCAIMem solution, extensive simulations and benchmarking against traditional SRAM are presented herein. The results presented herein demonstrate that MCAIMem significantly outperforms alternative on-chip memory designs in terms of area and energy efficiency. Specifically, the disclosed MCAIMem may reduce the area by 48% and energy consumption by 3.4* compared to SRAM designs, without incurring any accuracy loss.

[0034] Although various embodiments of hybrid memory modules may be disclosed herein in the context of specific applications, for example neural networks and / or Al chips, it is understood that the systems and methods disclosed herein may be used in any volatile memory application, including but not limited to general purpose computing, cache storage, high- throughput database systems, and other suitable applications such as, robot brains, autopilot systems in vehicles, super-resolution TV encoders / decoders, all forms of IOT / IOE, and mobile devices with integrated Al computational units.

[0035] 2T / 3T eDRAM gain cell circuit designs with full CMOS technology and operation, the two’s complement representation in DNNs, and the challenges and requirements for designing mixed SRAM and eDRAM cell memory for Al chips are discussed as follows.

[0036] Embedded DRAM cell and Sensing designs may include 3T and 2T eDRAM designs. Using fewer transistors per memory cell than traditional SRAM, 3T and 2T eDRAM designs permit a smaller area, higher density, and roughly 2* greater bit-cell density. The eDRAM gain cell is currently under active development, with the newest implementations seen in 7-10nm FinFET technology (see Choi, et al., IEEE Journal of Solid-State Circuits, 2015; Giterman, et al., IEEE Journal of Solid-State Circuits, 2018; Giterman, et al., IEEE Solid-State Circuits Letters,Docket # 047162-5382-OOWO2020; and Amat, et al., IEEE, 2018). Given the typically slow rate of technological scaling, the eDRAM gain cell retains its significance and utility, as illustrated in FIG. IB. For instance, as illustrated in FIG. IB, the retention time of eDRAM dramatically decreases as the feature size shrinks. The leakage current issue is alleviated in FinFET technology compared with planar technology, so the retention time could be improved in early-stage FinFET-nodes, e.g., 16nm (Guo, et al., Journal of Low Power Electronics, 2017). In sleep mode, eDRAM cells may exhibit lower cell leakage current than SRAMs, leading to reduced static power dissipation, encompassing both leakage and refresh power components. The cell write-margin of eDRAM cells is superior to SRAMs, as there is no contention between the access device and crosscoupled latch in a gain cell. However, conventional gain cells face short retention times due to the small storage capacitor and leakage currents that exponentially vary under Process- Voltage- Temperature (PVT) variations, causing higher refresh power dissipation and / or smaller read current. The former results from the more frequent refresh operation, while the latter is due to the faster cell voltage loss.

[0037] To comprehend eDRAM gain cells, consider a conventional 3T gain cell’s basic retention characteristics. In the 3T eDRAM cell as illustrated in FIG. 2A, PW represents the write access device, PS the storage device, and PR the read access device (Ichihashi, et al., 2005 Symposium on VLSI Circuits, 2005). In 3T eDRAM, the gate capacitor of PS is used to store the charge voltage for bit-1 or bit-0 representation. PS provides a smaller capacity compared to 1T1C eDRAM. As a result, 3T gain cells feature a decoupled read and write structure with separate Read Word-Line (RWL) and Read Bit-Line (RBL) for read access, and Write Word-Line (WWL) and Write Bit-Line (WBL) for write access. This leads to enhanced read and writemargins and flexibility in bit-cell design, allowing gain cells to scale well in future technology nodes. PW and PR are deactivated during data retention mode, leaving the storage node floating. The surrounding devices’ sub-threshold, gate, and junction leakages cause the floating voltage to change over time. Data retention time relies on the aggregated leakage current entering the storage node. Monte-Carlo simulations in SPICE, representing cell-to-cell variation in a 1 Mb memory macro using low power CMOS 45nm technology, display retention time variations as illustrated in FIG. 2A. With a read reference bias level of 0.65 V, both bit-1 voltage and bit- 0 voltage approach the read reference bias level at the same retention time.Docket # 047162-5382-OOWO

[0038] The innovative 2T gain cell design shown in FTG. 2B employs fewer transistors, featuring a single high-drive current NMOS read device driven by RWL and a PMOS write device that maintains the critical bit-1 voltage near VDD (Ichihashi, et al., 2005 Symposium on VLSI Circuits, 2005). FIG. 2B presents the 2T eDRAM cell, which has a substantially different structure and operational principle compared to its 3T counterpart in FIG. 2A. The prior 3T eDRAM cell used a PMOS device as the write access transistor to extend cell retention time by counteracting NMOS gate leakage with PMOS gate overlap and junction leakages. Nonetheless, under PVT variations, leakage compensation proves insufficient as the NMOS storage device’s inverted channel gate leakage dominates for bit-1, as illustrated FIG. 2A.

[0039] In the 2T eDRAM cell as illustrated in FIG. 2B, the read access transistor is substituted by the RWL signal, with a pre-charge level set at VDD. The storage transistor remains essentially off, making its gate leakage insignificant. Due to the absence of sub-threshold leakage through the read path, a low Vth transistor is proposed to improve read speed further. The stored cell exhibits asymmetry with a 0.65 V read reference bias, with bit-1 unaltered while bit-0 is prone to flipping to bit-1. The balanced P and N diffusion densities are another benefit of the proposed 2T asymmetric cell. The embodiments provided herein leverage this characteristic to minimize both static and dynamic energy consumption since bit-0 requires more energy to flip.

[0040] Embedded DRAM cell and sensing designs may include a 2T eDRAM Sense Amplifier design. In gain cells, the NMOS gate capacitor is used to store charge, rendering them sensitive to voltage changes. Directly accessing NMOS may cause a stored bit to flip. As a result, conventional gain cells may utilize a current sense amplifier to sense the storage node. For 2T cell designs, the RBL may exhibit a limited swing to avoid read failures caused by unselected cells’ leakage current. However, a smaller voltage swing results in a poorer read sensing margin. The asymmetric 2T gain cell further complicates the situation by utilizing a low Vth read device to achieve faster read speeds, while keeping the speed-critical bit-1 voltage close to VDD. To tackle this issue, a Current-mode Sense Amplifier (C-S / A) is employed in 2T eDRAM design, maintaining the RBL voltage near VDD during sensing and permitting the connection of multiple low Vth cells to a single RBL (Ichihashi, et al., 2005 Symposium on VLSI Circuits, 2005).Docket # 047162-5382-OOWO

[0041] As illustrated in FIG. 2C, the C-S / A includes a cross-coupled PMOS latch (Pl) and a pseudo-PMOS diode (P2) driven by the negative supply VBB, which is easily accessible on the chip for WWL under-driving. A negative WWL is utilized for a PMOS device to write a bit-1 into the cell without incurring a threshold voltage loss. Both PMOS pairs (Pl and P2) operate in saturation mode, providing improved matching. However, this C-S / A design is exclusively used for reading the storage bit in 2T eDRAM cells, as bit-0 still necessitates periodic write-back to avert bit flipping. Hence, an extra write circuit is used for the write operation, leading to inefficiency due to the small size of the 2T eDRAM cell and the substantial overhead needed for read / write circuits. The embodiments provided herein describe a modification to the 2T eDRAM cell that enhances its capacity and aligns it with the SRAM cell size. This slight alteration in the 2T eDRAM cell’s design simplifies the sensing circuit, making it compatible with both SRAM and 2T eDRAM. Unlike the conventional 2T which only leverages the asymmetric characteristics of 2T eDRAM for sensing in a small voltage swing with C-S / A, embodiments provided herein extend the reading process due to voltage sensing. The storage charge described in these embodiments has a larger voltage margin ranging from 0 to 0.8V for bit-0 and from 0.8 to 1.0V for bit- 1.

[0042] Two’s complement and a one-enhancement method in DNN representation are described as follows. In DNNs, the choice of data representation significantly impacts accuracy, computational complexity, and power consumption. Two’s complement representation is a popular format for signed integer values in DNNs because it simplifies arithmetic operations, particularly multiplication and addition. This format represents negative numbers as the two’s complement of the corresponding positive number’s binary representation, easing hardware implementation and reducing circuit complexity.

[0043] INT8 is regarded as the optimal representation for DNN inference, which maintains accurate results (Jouppi, et al., ISCA, 2021). The 8-bit two’s complement quantization is extensively adopted and outperforms other quantization techniques (Choi, et al., arXiv: 1805.06085, 2018). The embodiments provided herein utilize 8-bit two’s complement as the benchmark for designing the on-chip buffer.

[0044] In DNNs, the distribution of weights and activations typically follows a normal distribution, with most data values close to zero. FIG. 3 A illustrates the two’s complementDocket # 047162-5382-OOWO binary format of signed data. The two’s complement format is a method for encoding negative numbers in binary, facilitating signed arithmetic operations within digital systems. As illustrated in FIG. 3A, the first bit, known as the signed bit, determines whether the number is positive or negative. DNN data typically falls within a narrow range (e.g., [-50, 50]) (Nguyen, et al., IEEE Access, 2021). Therefore, negative values near zero possess 1 -dominant bits when represented in two’s complement, while the corresponding positive numbers exhibit 0-dominant bits. Converting 0-dominant bits involves flipping all data bits based on the signed bit. FIG. 3B illustrates the data histogram of weight data in the ResNet-50 deep learning network both before and after the application of a one-enhancement encoder. The histogram visually compares how the distribution of weights shift due to the encoding process. As illustrated in FIG. 3B, constructing the encoder requires one INV and seven XOR gates, transforming raw data into 1- dominant bit data. The one-enhancement encoder encodes DNN data into 1 -dominant bit data. The decoder processes the encoded data by flipping the bits according to the signed bit, thereby reconstructing the original data. FIG. 3C illustrates the bit histogram of weight data in the ResNet-50 deep learning network, again before and after using the one-enhancement encoder. FIG. 3C further illustrates a detailed view of how individual bit frequencies are affected by the encoding. Unlike conventional designs (Nguyen, et al., IEEE Access, 2021), the embodiments provided herein create 1 -dominant data to decrease refresh and static energy usage in mixed-cell memory design when storing DNN data. Further, the DNN data stored in on-chip memory undergoes encoding and decoding before computation.

[0045] The requirements of the one-enhancement encoder / decoder may include one or both of the signed bit acting as the control bit and dictating when encoding or decoding operations should be executed. The signed bit should be protected from errors. The embodiments described herein utilize 2T eDRAM to enhance area and energy efficiency. Nonetheless, 2T eDRAM demands more frequent refresh operations due to the accelerated cell voltage loss. To secure the signed bit, some embodiments disclosed herein allocate the signed bit to 6T SRAM and map some or all of the remaining bits to 2T eDRAM. As a result, the embodiments provided herein address the following challenges when designing mixed SRAM and eDRAM cells: compatibility, density and area efficiency, retention time and refresh rate, scalability, and reliability. Addressing compatibility in designing mixed SRAM and eDRAM cells ensured seamless integration of SRAM and eDRAM cells within a single memory architecture,Docket # 047162-5382-OOWO preserving compatibility with existing manufacturing processes. Addressing density and area efficiency in designing mixed SRAM and eDRAM cells achieved high memory density and area efficiency without sacrificing performance or increasing chip complexity. Addressing the intrinsic retention time and refresh rate of eDRAM cells and optimizing refresh rates minimized power consumption without compromising data integrity. Addressing scalability developed memory architectures that can be effortlessly scaled to accommodate diverse memory capacities and performance requirements for various Al applications, spanning from edge devices to data centers. Addressing reliability guaranteed the robustness and reliability of the mixed memory design under different operating conditions, especially in Al workloads involving frequent read and write operations.

[0046] The embodiments provided herein describe MCAIMem, which is an innovative on-chip mixed-cell memory design for Al chips. As illustrated in FIG. 4A, MCAIMem includes a one- enhancement encoder / decoder, a mixed SRAM / eDRAM cell memory, including mapping schemes and circuit-related designs, and the reference voltage controller responsible for lengthening the refresh operation duration. The on-chip MCAIMem may be a buffer for Al accelerators, storing both weights and activations during computation. Data transferred from off- chip DRAM may be retained in MCAIMem and subsequently employed by the DNN processing engine, which may range from traditional CPUs / GPUs to systolic arrays or computing-in- memory (CIM) architectures. The one-enhancement encoder / decoder enhances data encoding and decoding efficiency. The mixed SRAM / eDRAM cell memory combines different cell technologies to optimize performance and storage density. The reference voltage and refresh controller may ensure stable operation and compatibility.

[0047] Inbound / outbound data passes through the one-enhancement encoder module for encoding / decoding, as described herein. The encoded data is preserved in mixed SRAM / eDRAM cell memory, a combination of 6T SRAM and 2T eDRAM, aimed at minimizing the area footprint of on-chip memory. Owing to the inclusion of 2T eDRAM, the refresh controller provides periodic refresh operations, as further described herein.

[0048] FIG. 4A is a diagram of an example architecture of the MCAIMem 400. FIG. 4B is a flowchart illustrating an example process 401 of storing data to the MCAIMem 400. The MCAIMem 400 includes a one-enhancement encoder / decoder module 1, a mixed-memory cell 2,Docket # 047162-5382-OOWO and a refresh controller 3 operablely and electrically coupled to one another. The one- enhancement encoder / decoder module 1 may be a fully digital component. Features of these components are further discussed herein.

[0049] With reference to FIG. 4B, the MCAIMem 400 may receive a set of signed integers (at 402) to store data within the on-chip MCAIMem 400. For example, the MCAIMem 400, and preferably the one-enhancement encoder / decoder module 1 receives data to store within the mixed-cell memory 2 within the MCAIMem 400. The MCAIMem 400 may perform a one- enhancement process on the set of signed integers (at 404). For example, the one-enhancement encoder / decoder module 1 performs the one-enhancement process on the set of signed integers to encode the data. The one-enhancement process may include performing a bit flipping process on the received data corresponding to the set of signed integers, thereby encoding the data. For example, as illustrated in FIG. 4A, the one-enhancement encoder / decoder module 1 maps the signed / control bit to the one 6T SRAM cell and maps the following seven bits to 2T eDRAM cells. During the encoding process, the incoming data may undergo flipping based on the signed bit of the incoming data before being stored in the MCAIMem 400, as illustrated in FIG. 4A. By enhancing the raw bit data to predominantly 1 -bit values, the overall energy consumption of the memory may be reduced, as the cells are optimized to store and access 1 -bits more efficiently. The signed bit from the two’s complement representation may be utilized to perform this enhancement, as the signed bit is either 0 or 1 depending on the sign of the number. The encoder of the one-enhancement encoder / decoder module 1 modifies the input data such that more 1 -bits are present, while the decoder of the one-enhancement encoder / decoder module 1 reverses this process to recover the original data. A significant portion of DNN data is either 0 or values near 0. For instance, pruning results in 20-80% of the data being 0 (Li, et al., arXiv: 1608.08710, 2016). Since a majority of the data is close to zero, enhancing the representation to produce more bit-1 values may lead to power efficiency in memory systems without sacrificing data integrity or accuracy.

[0050] FIGs. 5A and 5B illustrate a bit statistic histogram comparison for weights and activations pre-function and post-function of the one-enhancement encoder / decoder module 1. As illustrated in FIGs. 5A and 5B, the 6th, 5th, and 4th bits mostly convert to bit-1, making it highly efficient to map them to 2T eDRAM cells. The Oth, 1st, 2nd, and 3rd bits continue toDocket # 047162-5382-OOWO contain a considerable number of bit-Os. When using 2T eDRAM to store such bits, retention errors might occur. However, due to the asymmetric nature of 2T eDRAM, flipping errors from 0 to 1 are far more common than flipping errors from 1 to 0, and due to the predominance of bit- 1 in the encoded data, the overall probability of flipping errors is reduced. These errors can potentially impact the output of DNN applications. The implications of these errors on the output of DNN applications are evaluated herein to ensure these errors do not adversely affect the DNN application outcomes.

[0051] With respect to a mixed cell mapping scheme: using 2T eDRAM may lead to errors resulting from its short retention time. These errors may arise when storing DNN data in MCAIMem 400. Based on the one-enhancement encoder / decoder module 1, the control bit is of utmost importance since an error in the control bit would cause errors in all the remaining bits. Consequently, it is important to ensure the control bit is well protected when storing DNN data in 2T eDRAM, while allowing for approximation in the remaining bits. The bit mapping may be as follows: 1) map the control bit to 6T SRAM, and 2) map the 7 least significant bits (LSBs) to 2T eDRAM. The reasons for selecting one SRAM and seven 2T eDRAM units is further discussed herein.

[0052] As illustrated in FIG. 4A, one 6T SRAM cell is allocated for the signed / control bit, while the following 2T eDRAM cells are allocated for the following seven bits. Incoming DNN data is encoded by the encoder of the one-enhancement encoder / decoder module 1. Returning to FIG. 4B, the MCAIMem 400 may store the one-enhanced set of signed integers (at 406). For example, the MCAIMem 400 stores the encoded data in the mixed-cell array, that is, the mixed-cell memory 2. The signed bit / control bit is securely stored in the 6T SRAM, while the remaining flipped bits are stored in the cells of the 2T eDRAM, as shown in FIG. 4 A. This memory mapping approach ensures the safety of the signed / control bit in the 6T SRAM.

[0053] Returning to FIG. 4B, to prevent data flipping, the MCAIMem 400 may refresh the one- enhanced set of signed integers (at 408). The MCAIMem 400, and preferably the refresh controller 3, performs periodic refresh operations on the flipped bits stored in the cells of the 2T eDRAM. The refresh controller 3 may perform the refresh operation at a predetermined frequency to prevent data flipping. This refresh operation maintains the dominance of bit-1 in the majority (around 80%) of DNN data. Due to the characteristics of 2T eDRAM, storing bit-1Docket # 047162-5382-OOWO consumes less energy than storing bit-O. Therefore, static energy savings may be achieved by using the encoder of the one-enhancement encoder / decoder 1 in conjunction with asymmetric 2T eDRAM.

[0054] The mixed-cell memory 2 of MCAIMem 400 includes one 6T SRAM cell and seven 2T eDRAM cells, as illustrated in FIG. 4A. An asymmetric 2T was chosen instead of the asymmetric 3T as asymmetric 3T does not clearly keep the charge like 2T, where the 3T tends to consume the same power when storing 0 / 1. The pitch matching provides more storage capacitors and makes 2T have a longer retention. To integrate these cells, the circuit designs of both 6T SRAM and 2T eDRAM are modified. Additionally, a voltage sense amplifier may be used for both 6T SRAM and 2T eDRAM. The details of these modifications are discussed herein.

[0055] MCAIMem 400 provides enhanced retention time of asymmetric 2T eDRAM cell. Combining the designs of 6T SRAM and 2T eDRAM created challenges regarding pitch lane mismatches and the need for a mixed sense amplifier suitable for both 6T SRAM and 2T eDRAM. This is due to the considerably smaller size of the 2T eDRAM compared to the 6T SRAM. To resolve the pitch lane issue, the size of the 2T eDRAM’s storage transistor is adjusted. As illustrated in FIG. 6C, pitch lane mismatches may occur when designing the cell layout for 6T SRAM and 2T eDRAM. The 2T eDRAM cell may occupy 60% of the area compared to an SRAM cell. Therefore, the width of the 2T eDRAM may be increased up to 4 x to align with the design of the 6T SRAM cell.

[0056] Pitch lane matching may be particularly advantageous for example in alternative embodiments where different proportions of 2T eDRAM to 6T SRAM are used. For example, in some embodiments, instead of storing only the signed bit in SRAM and the remaining bits in 2T eDRAM, a hybrid memory may include an additional one, two, three, four, five, or six additional SRAM bits in place of the 2T eDRAM bits. Doing so may improve the accuracy of the retained values, at the cost of additional power and surface area. For example, in one embodiment, numbers greater than 64 and up to 96, after processing through an enhanced encoder, are represented as OOlx xxxx (MSB->LSB). The second bit from the left, which is 0, may easily flip to 1. To protect this bit, an additional SRAM may be used, resulting in a total of two SRAMs used. Further, for numbers greater than 96 and less than 112, after encoding, the representation becomes 0001 xxxx. The third bit from the left, initially 0, may also easily flip to 1. Therefore,Docket # 047162-5382-OOWO one more SRAM may be used to protect this bit, bringing the total to three SRAMs. Increasing the number of SRAMs in the most significant bits may further enhance the accuracy of the stored values. Comparative data of using different mixes of SRAM and eDRAM cells is shown in FIG. 10A

[0057] The 2T eDRAM includes two primary components: the access transistor and the storage node. The gate capacitor (Cg) in the storage node stores the charge voltage representing bit-1 or bit-0, as illustrated in FIG. 6A. The capacity of the NMOS gate may be defined as Cg <x WLCox. By increasing the width of the NMOS gate, not only is the storage node’s capacity enhanced, but also the 2T eDRAM’s retention time is improved. FIG. 6B shows the results of a SPICE simulation of an eDRAM design using CMOS 45 nm technology, showing the storage voltage for a stored bit-0 in a standard 2T eDRAM (WN = Wp, as shown in FIG. 6C) as compared to the extended 2T eDRAM (WN = 4Wp, as shown in FIG. 6C), which is pitch lane matched to the 6T SRAM . When storing bit-0, the retention time is significantly extended in the extended 2T eDRAM. For example, the time required for the charge to change from 0.18V to 0.8V doubles.

[0058] Moreover, increasing the storage node’s capacity offers additional benefits. It makes the 2T eDRAM more robust against read-disturb effects. As such, the RWL and RBL in the 2T eDRAM may be removed and connected directly to VDD. The NMOS transistor may function solely as a capacitor for storage. The gate leakage from VDD of the storage node, along with the gate leakage and junction leakage of the write transistor, refills the storage node’s charge to bit- 1. As a result, the asymmetric characteristic of 2T eDRAM remains intact. With this design, bit- 1s may be stored without retention time, while bit-Os may need periodic refresh operations to maintain their discharged state. Thus, the drain and source of the storage node may be directly connected to VDD, as illustrated in FIG. 6A. The pull-up leakage current is utilized to sustain the ‘ 1’ bit and to store the ‘0’ bit, thereby eliminating the need for changes in doping and gate oxide thickness typically required for Low Voltage Threshold (LVT) devices. As such, the MCAIMem 400 design renders such modifications unnecessary.

[0059] The SRAM cell design is adapted to mixed memory cells: For example, while significant modifications were made to the storage node of the 2T eDRAM, the PMOS access transistor from the conventional 2T eDRAM design is retained to ensure that only bit-0 changes while bit- 1 remains unchanged. VDD+0.4V is applied to the drain and source of the storage node, asDocket # 047162-5382-OOWO illustrated in FIG. 6A. to minimize the subthreshold leakage of the PMOS and ensure that the pull-down leakage path is always lower than the pull-up leakage path. In some cases, using PMOS transistors as the access transistor may conflict with the design of the access transistor in 6T SRAM.

[0060] To address this issue, the SRAM cell design as illustrated in FIG. 7A is modified by changing the access transistor in the SRAM to PMOS as well. By aligning the access transistor types in both memory cells, the integration of 6T SRAM and 2T eDRAM designs are facilitated while maintaining the desired functionality and performance. The key change involves the positioning of the access transistors in layout, in line with logic design rules. This adjustment optimizes the design without impacting the footprint of the SRAM cells.

[0061] By modifying the access transistor in a 6T SRAM bit cell (as illustrated in FIG. 8 A), a higher read static noise margin (SNM) of lOOmV with a PMOS access transistor (red line) compared to 90mV with an NMOS transistor (black line) is observed. PMOS transistors have lower writeability. As node QB discharges and Vgsdecreases, the transistor weakens and shuts off when QB dips below the threshold voltage, leading to a constrained write margin of 30mV at FS corner (green line). This issue may be mitigated by applying a small negative voltage during the 6T SRAM write process (Nabavi, et al., IEEE Journal of Solid-State Circuits, 2017). With a -0. IV on the word line (WL), the write yield of the PMOS access transistor increases to match that of an NMOS transistor as illustrated in FIG. 8B.

[0062] The MCAIMem 400 includes a voltage sense amplifier circuit for both SRAM and 2T eDRAM. The 2T eDRAM decouples the read and write paths, which means that separate circuits are required for read and write operations. Additionally, the short retention time of 2T eDRAM necessitates periodic refresh operations to maintain the data. One of the major challenges in designing mixed cells is providing a mixed sense amplifier that caters to both 6T SRAM and 2T eDRAM.

[0063] In 2T eDRAM, a current sense amplifier is used to detect small gains in the read path without disturbing the data in the storage node. The refresh operation demands that the read data be written back to the storage node, resulting in substantial peripheral circuit overhead. By increasing the width of the 2T eDRAM by 4*as shown in FIG. 6C, the design may resist read-Docket # 047162-5382-OOWO disturb. Therefore, a voltage sense amplifier, as illustrated in FIG. 7C, is included for both 6T SRAM and 2T eDRAM. The voltage sense amplifier enables read and write operations for both 2T eDRAM and 6T SRAM to be identical. The primary distinction between 6T SRAM and 2T eDRAM when connected to the voltage sense amplifier is that both the BL and BLB of the 6T SRAM are connected. In contrast, for the 2T eDRAM, only one BL connects to the sense amplifier, while the BLB of the sense amplifier links to the reference voltage (VREF), as illustrated in FIG. 7B.

[0064] Not only does this design simplify read and write operations, but the use of a voltage sense amplifier also enables writing data back to the 2T eDRAM storage node during read operations. This streamlines the refresh process, as only one read operation is needed to perform the refresh, rather than the conventional sequence of read and write back operations in standard 2T eDRAM designs.

[0065] FIG. 9A is a timing diagram of the write operation using the voltage sense amplifier (VS A). The process begins by applying voltage to the bit-line out (BLO1), followed by enabling the sense amplifier through the enable signal (EN). This action causes the bit-line to charge or discharge. When the word-line (WL) is activated, the bit-line voltage alters the data within the 6T SRAM or 2T eDRAM. For the SRAM, the PMOS access transistor in the 6T SRAM is weaker than the storage node for the write operation to be successful. In the case of the 2T eDRAM, the charge or discharge of the storage node occurs similarly to the SRAM.

[0066] FIG. 9B illustrates the read operation of the VSA. To initialize the sense amplifier for the read operation, the precharge is enabled, charging BLO1 and BLO2 to 1. For the 2T eDRAM, a reference voltage may be applied to the bit-line bar (BLB). This reference voltage (VREF) is employed to compare the voltage in the storage node and determine the data output of BLO1. Once the WL and EN in the sense amplifier are enabled, the storage node may either recharge or discharge the bit-line. If the bit-line (BL) voltage is greater than VREF within the sense amplifier, BLO1 is set to 1; if the BL voltage is less than VREF, BLO1 is set to 0. The read operation may potentially destroy the data stored in the 2T eDRAM. As a result, it is crucial to disable the WB in the sense amplifier to recharge the storage node in the 2T eDRAM. For the 6T SRAM, the BLB of the VSA is connected to the BLB of the SRAM cell.Docket # 047162-5382-OOWO

[0067] As illustrated in FIG. 9B, the revised 2T eDRAM’s read operation allows for the deactivation of the write-back (WB) signal. Due to the bitline voltage, the storage node may be charged or recharged, rendering the MCAIMem refresh operation as simple as executing a read operation.

[0068] The mixed-cell memory of MCAIMem 400 incorporates one or more 2T eDRAM cells, which require periodic refresh operations. The periodic refresh method (Baek, et al., IEEE Transactions on Computers, 2013), also known as the global refresh operation may be used. In this approach, the refresh controller 3 may perform a refresh operation on each row in the mixedcell memory array of MCAIMem 400 within 12.57ps. To elaborate, the ordinary refresh cycle interval is calculated by dividing the refresh time by the number of rows. While the static power consumption for storing a DNN’s data is significantly reduced due to the encoder of the one- enhancement encoder / decoder 1, bit-0 requires frequent refresh operations to ensure the safety of the DNN’s data. The refresh controller 3 is responsible for determining the reference voltage for the 2T eDRAM sense amplifier, which aids in extending the refresh period and reducing the dynamic refresh energy for bit-0 in the DNN’s data.

[0069] To retrieve the stored data (i.e., the original data) within the MCAIMem 400, the example process 401 may essentially be performed in reverse. For example, following the mapping process described herein, the MCAIMem 400 retrieves the encoded data from the mixed-cell memory 2. The decoder of the one-enhancement encoder / decoder module 1 flips the encoded data based on the signed / control bit. For example, based on the signed / control bit stored within the one 6T SRAM cell, the decoder flips the encoded data within the remaining 2T eDRAM cells to decode the data and recover the original data.Experimental Examples

[0070] An assessment was performed on the performance of the one-enhancement encoder / decoder module (e.g., the one-enhancement encoder / decoder module 1). The experimental assessment was performed after conducting synthesis on the 45nm technology node. As the one-enhancement encoder / decoder module is a fully digital component, the rtl and synthesis were implemented with Synopsys’ de compile and the prime time tool based on the simulated random input testbench with the assumption of maximum clock frequency of IGhz isDocket # 047162-5382-OOWO used to evaluate the power consumption. The power consumption of the one-enhancement encoder / decoder module, measured at 1.35xlO"2mW, constitutes a mere 0.007% of the total memory power, rendering its impact negligible. In terms of area, the one-enhancement encoder / decoder module occupies 35.2« / »2, which is 0.004% of the 108KB memory’s size. These metrics underscore the module’s inconsequential influence on both power usage and spatial requirements, especially when juxtaposed with the vast expanse of memory cells. Moreover, the delay associated with the one-enhancement encoder / decoder module stands at 0.23ns. Even with a clock period of Ins (corresponding to a clock frequency of 1GHz), there is a comfortable slack of 0.67ns, ensuring the absence of timing violations. As such, the delay of the one-enhancement encoder / decoder module does not pose a threat to the overall performance of the on-chip MCAIMem system.

[0071] With respect to the effects of retention error on the Al chip outcomes, DNNs are known for their robustness, with errors typically being minor and only occurring in the LSBs (Li, et al., ACM, 2017). In MCAIMem, the memory configuration includes a mix of SRAMs and 2T eDRAMs. In the mixed-cell memory design, potential errors may originate from the 2T eDRAM due to retention errors, thus necessitating an exploration of their impact on DNN applications. The 2T eDRAM demonstrates a bit-0 to bit-1 flip in under 1% of cases before 12.57 ps and in over 10% post 12.8 ps, with no observed errors for bit-1, as illustrated in FIG. 1 IB. This informs the utilization of an error injection method to evaluate the flipping error rate and discern its effect on the accuracy of DNN applications, which subsequently influences the refresh period of the disclosed MCAIMem. Notably, retention time issues are exclusive to the 2T eDRAM, not affecting the SRAM. Thus, errors may be deliberately injected into both the weight and activation of DNN data before every computation, allowing for evaluation of the cumulative effect of the errors. That is, the errors from the first injection accumulate and affect the second injection. Therefore, before computation, errors are injected into the data currently stored in memory, which is no longer the original data, thereby resulting in the cumulative effect of the errors. As such, errors may be initially injected into the DNN, absent the one-enhancement encoder / decoder, where only bit-Os are flipped at a predefined error rate. Alternately, errors may be injected into bit-0 post-application of the one-enhancement encoder and prior to decoding the data, with the flipping error rate spanning from 1% to 10%.Docket # 047162-5382-OOWO

[0072] Simulations were performed on several CNNs, such as LeNet, VGG1 1, VGG16, AlexNet, and ResNet50 (see Krizhevsky, et al., NIPS, 2012; LeCun, et al., 2015; Szegedy, et al., IEEE, 2015; and He, et al., IEEE, 2016), using datasets like MNIST, CIFAR10, CIFAR100, and ImageNet. In language modeling, I-BERT (Kim, et al., PMLR, 2021), the integer version of BERT, with the GLUE datasets (Wang, et al., arXiv: 1804.07461, 2018) was utilized. As for generative modeling, the quantized version of CycleGAN (Zhu, et al., IEEE, 2017) with the horse2zebra dataset (Wang, et al., European Conference on Computer Vision, 2020) was used. This comprehensive evaluation revealed the extent to which different retention error levels affect DNN accuracy and the effectiveness of the one-enhancement method. For each error rate specified, a comparison of DNN accuracy was made when MCAIMem was used with and without the one-enhancement encoding. For Generative Adversarial Networks (GANs), direct measurement of output accuracy is not possible. Instead, the mean relative error was used as a metric to quantify the differences between the enhanced GAN and the original model. This approach enabled assessment of the effectiveness of the one-enhancement method in mitigating the impact of retention errors on DNN accuracy. In the context of I-BERT, a language model, changes in perplexity were used as the primary metric, as top-1 accuracy was not applicable. As illustrated in FIG. 10A, the ratio of SRAM cells to eDRAM cells impacts the final DNN accuracy significantly, especially since the most significant bits (MSBs) are highly sensitive to errors, even with a bit error rate (BER) of 1%. FIG. 10A illustrates that without the one- enhancement encoder / decoder, DNN accuracy drastically drops to zero across various networks. This decrease is attributed to the protection of only the MSBs in SRAM, leaving the other bits vulnerable to retention errors. With the implementation of the one-enhancement encoder / decoder, most MSBs are converted to one bits, making them less likely to flip, whereas a minority of the less significant bits (LSBs) that remain as zero bits may be subject to retention errors. Consequently, an error injection of up to 1% is tolerable for applications like ImageNet on AlexNet / ResNet50, GLUE on I-BERT, and horse2zebra on CycleGAN. Moreover, datasets such as MNIST and CIFARlO / 100 show even greater error resilience, tolerating retention errors of up to 10%. Although the one-enhancement method introduces significant bit errors into the MSB of weight and activation values, reducing inference task accuracy, it is advantageous for prolonging eDRAM retention time. Despite introducing bit errors, these predominantly affect the LSB, minimally impacting the accuracy of the inference process. For I-BERT and CycleGANDocket # 047162-5382-OOWO applications, maintaining accuracy comparable to the standard requires a configuration of seven SRAM and one 2T eDRAM units. As illustrated in FIG. 10B, the total power consumption for a hybrid configuration of seven SRAM and one 2T eDRAM exceeds that of conventional SRAM due to the refresh operations needed for 2T eDRAM. Additionally, the transition from a BER of 10% to 1% occurs in less than 0.3 ps, indicating that the MCAIMem design of one 6T SRAM and seven 2T eDRAM offers the highest power efficiency. Therefore, the MCAIMem design is evaluated further. In the realm of DNN applications, ensuring the accuracy of the output is paramount, surpassing the importance of hardware performance and energy efficiency. Thus, the MCAIMem design prioritizes maintaining DNN output accuracy, accommodating a maximum retention error of up to 1% in the disclosed mixed-cell architecture.

[0073] Certain embodiments disclosed herein relate to a refresh period extension with an adaptive VREF. As discussed in more detail above, bit-0 tends to flip to bit-1 over time, and only bit-0 is subject to retention errors. To maintain DNN output accuracy with the one-enhancement encoder, the maximum allowable retention error is 1%. As a result, a refresh time is determined that preserves DNN output accuracy by developing an error model based on bit-0’ s retention time.

[0074] In 2T eDRAM, leakage current causes bit-0 to have a propensity to flip to bit-1 after a specific duration. This leads to variations in bit-0 readings, depending on the 2T eDRAM access time. To compute the 0-to-l flipping probability, a Monte Carlo simulation was executed, generating a multitude of variation samples of 2T eDRAM storing bit-0. The number of flipped bits concerning the total number of 2T eDRAM samples, was counted considering the access time and specific reference voltage (VREF), as illustrated in FIG. 11 A. This error flipping model assists in identifying the optimal VREF for achieving a balance between robustness, retention time, and energy efficiency in MCAIMem.

[0075] The Monte Carlo simulation was conducted 100,000 times at a temperature of 85°C, which reflects typical desktop and server working conditions, where temperatures range from 25°C -85°C (Kim, et al., ACM SIGARCH, 2014). This involved assessing data shifts in the storage node and varying the access time between 0 and 20 microseconds while simultaneously reading the data. The read-out data was compared to the reference voltage (VREF) to simulate the sense amplifier model, which determined whether the output data was 1 or 0. FIG. 1 IBDocket # 047162-5382-OOWO illustrates that with VREF at 0.5, a 1% flipping probability initiates at 1 .3 microseconds. Conversely, with VREF at 0.8, a 1% flipping probability starts at 12.57 microseconds. The graph indicates that the flipping probability slope is steep, meaning that extending the refresh period based on a specific VREF yields minimal refresh power reduction. Nevertheless, adjusting VREF lengthens the required refresh period. As a result, a VREF of 0.8V may be used in some embodiments to maximize bit-0’ s refresh period and minimize dynamic refresh operations in mixed-cell memory.

[0076] The disclosed MCAIMem design relates to Al chips intended for server and desktop applications, operating within a temperature range of 25°C to 85°C; however, it should be understood that the MCAIMem design is not limited to such chips. The experiments and evaluations provided herein do not consider voltage variations, concentrating solely on process variations through Monte Carlo simulation. The evaluation of the design utilizes a circuit simulation, followed by an examination of system simulation. The latter particularly explores DNN applications powered by the MCAIMem design.

[0077] In the circuit evaluation, layouts for 1MB 6T SRAM, 2T eDRAM, and the mixed-cell memory using CMOS 45nm technology were created. The chip area for these embedded RAMs based on their layout sizes was calculated and the sizes were compared. Additionally, the SPICE models of these memories were extracted, and the post-simulations were performed to analyze the static power, read, and write operations for each memory type. Table 2 below summarizes the characterization results.Table 2

[0078] As illustrated in FIG. 12A, the mixed SRAM and eDRAM design reduces the area size by 48% compared to SRAM memory alone. In this design, both 6T SRAM and 2T eDRAM utilize the same Voltage Sense Amplifier (VSA), as illustrated in FIG. 7A and detailed in theDocket # 047162-5382-OOWO procedure of FIGs. 9A and 9B. The overhead ratio of VSAs, for an 8-bit word where 32 cells share the same bit-line, was calculated as (3.4225*2.6575) / (3.4225*34.275)=8%. In the MCAIMem design, the timing overhead for read / write operations in SRAM was approximately Ins. In contrast, eDRAM operations, including recharge and discharge cycles, require a specific duration, also around Ins, in addition to a peripheral delay of Ins. Consequently, the overall speed of the memory system is constrained by the eDRAM’s slower speed, even though SRAM is capable of faster read / write operations. During read operations, both reading and writing back occur within a timeframe of 2ns. For write operations, data is directly written to eDRAM, incurring a time cost of Ins. During circuit simulation, the asymmetric characteristic of 2T eDRAM causes the stored data value to impact static and access power. When all bit data is 1, eDRAM consumes less power as the leakage is substantially lower when the storage node is at VDD. The gate leakage from VDD to the storage node is minimal, with the main leakage now being the subthreshold current of the PMOS. This current is small, as a delta of 0.4V is applied to the gate of the PMOS access transistor. If all bit data is 0, a higher gate leakage current from VDD attempts to recharge the storage nodes to bit-1. Consequently, the one-enhancement technique increases the number of bit- 1 bits, which is essential for reducing static power in 2T eDRAM.

[0079] In some embodiments as disclosed above, the mixed-cell memory includes one 6T SRAM and seven 2T eDRAMs. Static power originates from both the SRAM and eDRAM, yet it can be reduced by 3-6* compared to SRAM alone. Regarding read and write operations, the 6T SRAM is mostly balanced, while 2T eDRAM continues to display asymmetric characteristics. When reading a bit-1, the initial BL is VDD, so there is no change in the sense amplifier, leading to low energy consumption. Conversely, when reading a bit-0, the storage node is recharged to 0, with the current from the storage node being the main contributor to energy consumption.

[0080] This evaluation examines the impact of MCAIMem on various DNN applications, including convolutional neural networks (CNNs), the I-BERT language model, and the Cycle- GAN generative network. The SCALE-Sim is adapted (Samajdar, et al., arXiv, 2018) to assess the static and dynamic energy consumption of different memory devices within the configurations of Eyeriss (Chen, et al., JSSCC, 2016) and Google’s TPUvl (Jouppi, et al., ISCA 2021). To align the power model with the specific memory requirements of these devices, theDocket # 047162-5382-OOWO embedded RAM power model was adjusted based on their SRAM needs. For Eyeriss, with its 108KB SRAM requirement, the power model was scaled down to one-tenth for the standard 1MB memory configuration. For the Google TPUvl, requiring 8MB, the model was scaled up by a factor of eight. The RRAM model from Prabhu, et al., IEEE Journal of Solid-State Circuits, 2022 was utilized, assuming the use of RRAM for both weights and activations as the on-chip buffer, reflecting memory size adjustments for both Eyeriss and TPUvl. Notably, no static power consumption was assumed for RRAM due to its non-volatile nature, which allows toggling on and off without data loss, and focused on read and write energy per byte. Details of the simulation configurations are provided in Table 3 below.Table 3

[0081] In this simulation-centric study, the computation time for each device configuration was extracted, operating under an assumed clock frequency of 100 MHz. Subsequent to determining the computation time for each memory type, respective power models were applied to calculate the final static and dynamic energy. The 6T SRAM and conventional 2T eDRAM (sans the one- enhancement encoder / decoder) were utilized as the baseline comparison. The evaluation rigorously focused on the performance of the on-chip buffer, deliberately excluding the energy consumption from MAC operations. Additionally, the impact of softmax, other nonlinear operations, and off-chip memory access was not considered in this analysis. The choice of a 100MHz clock frequency was based on the lower operational clock frequencies typically found in Al accelerators, exemplified by Eyeriss operating at 100MHz (Chen, et al., JSSCC, 2016) and TPUvl at 700MHz (Jouppi, et al., ISCA, 2021), highlighting alignment with industry standards. This chosen clock frequency not only determines the time to access and hold memory across layers but also proves imperative given the usage of eDRAM, which necessitates a refresh operation to safeguard data. The clock frequency thereby becomes vital in estimating the requisite number of refresh operations amidst Al accelerator computations. Employing SCALE-Docket # 047162-5382-OOWOSIM enables the quantification of the number of clock cycles, and owing to the systolic array’s design, each clock cycle concurrently facilitates MAC and memory access, thereby simplifying the tally of on-chip memory accesses. FIGs. 13 A and 13B delineate the minimum power savings attainable. Interestingly, with the adoption of a swifter clock frequency, data retention time per layer is truncated, possibly culminating in diminished power savings due to a decrease in the number of refresh operations while computations are in progress.

[0082] When it comes to static power, SRAM energy consumption is higher than both the 2T eDRAM and the disclosed mixed-cell memory. Although the mixed-cell memory has a higher static energy consumption than the 2T eDRAM, it performs better than SRAM. With a 1 :7 SRAM:eDRAM ratio, the fixed energy overhead of SRAM in the mixed-cell memory accounts for 76.5% of the total consumption. Further details may be found in FIG. 13A.

[0083] As for refresh power, SRAM does not require refresh operations, while 2T eDRAM and the mixed-cell memory do. Adjusting the reference voltage (VREF = 0.8) may help increase the refresh period and reduce the frequency of refresh operations. Regarding total energy consumption, which encompasses both static and dynamic energy utilization throughout the inference process, eDRAM delivers a compact area footprint yet does not excel in overarching energy consumption due to its refresh energy requirements. Contrarily, the mixed-cell memory affords advantages in both area footprint minimization and energy consumption reduction, attaining an energy efficiency that is 3.4 X superior to 6T SRAM, as illustrated in FIG. 13A.RRAM lags in energy efficiency, being over 100 X higher than SRAM, attributed to its requisite for substantial write operations. The energy use of RRAM is higher in spite of its non-volatility because of the higher energy requirement for each write operation.

[0084] Given that the on-chip buffer contributes to 42.5% of the power consumption in Eyeriss (Chen, et al., JSSCC, 2016) and 37% in TPUvl (Jouppi, et al., Proceedings of the 44th annual international symposium on computer architecture, 2017), utilizing an MCAlMem configuration with VREF =0.8 allows the performance-per-watt to attain gains between 35.4% and a peak of 43.2%, surpassing the efficiency of an on-chip buffer that employs SRAM, as illustrated in FIG. 13B. Varying the clock frequency affects the retention time of data stored in on-chip memory, leading to reduced static power consumption. As illustrated in FIG. 14, increasing the clockDocket # 047162-5382-OOWO frequency can decrease total energy consumption by up to a factor of 5*. Consequently, MCAIMem emerges as a compelling solution, potentially paving the way for innovations in efficient Al memory design.

[0085] Deep Neural Networks (DNNs) require a vast number of parameters to achieve superior performance, leading to increased memory demands. Addressing the increased requirements for the on-chip data buffer and data movement is essential for enhancing DNN accelerator performance. Chen, et al., JSSCC, 2016 demonstrated that off-chip DRAM access consumes 200 x more energy and has a longer access time compared to ALU. As a result, optimizing the on-chip buffer has become a primary challenge in boosting DNN accelerator throughput. The key question is how to maximize on-chip memory capacity and minimize off-chip access to improve energy efficiency in DNN accelerators.

[0086] DaDianNao (Chen, et al., IEEE, 2014) suggests replacing SRAM with fully eDRAM (1T1C) in conventional DNN accelerators to increase on-chip buffer capacity significantly. However, this approach necessitates periodic refreshes to maintain DNN data, leading to substantial energy consumption — accounting for 38.3% of the total DNN accelerator power usage. RANA (Tu, et al., ISCA, 2018), a more recent technique, exploits the shorter activation data lifetime compared to eDRAM retention time, allowing the elimination of unnecessary refresh operations. As DNN applications evolve, this observation may become less applicable, resulting in increased activation data and potential violations of the activation data lifetime constraint.

[0087] Computing-in-memory (CIM) has been proposed as an alternative to traditional DNN accelerators, aiming for improved throughput. Techniques such as the 4T Dual eDRAM array (Yoo, et al., ISLPED, 2019), which employs two 2T eDRAMs, and DualPIM (Jung, et al., IEEE, 2022), which utilizes hybrid SRAM and eDRAM configurations as computation nodes, have been developed. Additionally, recent eDRAM node optimization (Ha, et al., IEEE, 2022) focuses on reducing leakage and enhancing robustness in CIM. While these approaches demonstrate substantial performance and energy savings, the need for on-chip buffers remains.

[0088] Aiming for energy efficiency, some published applications implement voltage scaling. Chang, et al., IEEE, 2011 initially employed a hybrid of 10T / 8T SRAM, and subsequently, aDocket # 047162-5382-OOWO combination of 8T SRAM and 3T eDRAM was adopted for video applications (Kazimirsky, et al., IEEE, 2017). These applications benefit from the brief duration of video frames, during which human eyes are unlikely to notice minor changes. Data is bifurcated into two segments: the most significant bits (MSBs) and the less significant bits (LSBs). MSBs are stored in cell designs that are robust at lower voltages, enhancing reliability, while LSBs are allocated to less robust cells that offer power savings. This strategy allows for power reduction in LSBs without significantly impacting the quality of video frames, ensuring functionality is maintained. However, the application of this technique in machine learning contexts raises questions about how approximation affects machine recognition capabilities.

[0089] A recent study by Yin et al. explored the use of a hybrid SRAM and ROM architecture within residual deep learning networks (Yin, et al., IEEE Journal of Solid-State Circuits, 2023). Employing ROM for storing weights has shown significant power savings. However, a major drawback of using ROM is its inflexibility, as it cannot be modified after fabrication. In contrast, the 2T eDRAM offers the advantage of flexibility, and with the implementation of the One- enhancement architecture, storing data with a value of ‘ 1’ does not significantly increase power consumption, even considering the necessity for refresh operations. The hybrid use of SRAM and 2T eDRAM could also play a crucial role in Compute-in-Memory (CIM) applications. It is worth noting that both the DualPIM and ROM-based CIM approaches discussed in Yin et al. focus primarily on weight data, with less attention given to the handling of activation data. The comparisons of these hybrid solutions are summarized in Table 4 below. The SRAM data is taken from Chang, et al., IEEE, 2011. The SRAM-eDRAM data is taken from Kazimirsky, et al., IEEE, 2017. The SRAM-ROM data is taken from Yin, et al., IEEE Journal of Solid-State Circuits, 2023.Docket # 047162-5382-OOWOTable 4

[0090] Furthermore, a study called ZEM (Nguyen, et al., IEEE Access, 2021) explores the asymmetric characteristics of DNN data to extend DNN data retention time in off-chip DRAM, significantly reducing off-chip DRAM power. However, the embodiments described herein primarily aim to decrease off-chip DRAM power during DNN application processing rather than tackling the core challenge of improving DNN accelerator performance by minimizing off-chip DRAM access. The embodiments described herein target the on-chip buffer by creating a mixed SRAM and eDRAM cell design to minimize the area and energy consumption of the on-chip buffer. This approach holds promise for the design of on-chip buffers in next-generation DNN accelerators.

[0091] The embodiments described herein introduce MCAIMem, an innovative area and energyefficient Al memory design that utilizes a mixed CMOS memory cell design, comprising both SRAM and eDRAM cells. The embodiments optimize the ratio of SRAM / eDRAM cells to achieve reduced area and capitalize on DNN’s data representation and asymmetric eDRAM cells for lower energy consumption. Experimental results demonstrate that the MCAIMem design may decrease the area by 48% and energy consumption by 3.4* compared to conventional SRAM designs, without sacrificing accuracy. Further, the embodiments highlight the potential of mixed CMOS memory cells and asymmetric 2T eDRAM cell implementations in attaining an optimized balance between performance, area, and energy consumption for Al memory designs. Moreover, the mixed CMOS cell memory design, MCAIMem, provides a promising solution and has the potential to become a new standard for efficient Al memory design.Docket # 047162-5382-OOWO

[0092] The embodiments provide the first mixed 6T SRAM and 2T eDRAM cells for on-chip Al memory. The embodiments modify 2T eDRAM cells to align with SRAM cells and enhance capacity for longer retention times. The mixed memory cells significantly reduce the on-chip memory footprint for Al accelerators. The embodiments provided herein describe a common voltage sense amplifier (CVSA), which can be used for both SRAM and 2T eDRAM cells. By controlling the reference voltage of the CVSA, the refresh period of 2T eDRAM can be extended, reducing MCAIMem’s dynamic refresh energy. The asymmetric 2T eDRAM is exploited, where storing bit-1 consumes less energy than bit-0. Combining a one-enhancement encoder / decoder for DNN data addresses eDRAM reliability issues, such as refresh rate and retention time, significantly reducing MCAIMem’s static power. The MCAIMem reduces the area consumption by 48% and improves energy efficiency by 3.4* in on-chip Al memory systems, as illustrated in FIG. 15, by blending the strengths of 6T SRAM and 2T eDRAM to create a high-performance, energy-efficient, and compact hybrid memory solution.

[0093] The various embodiments described above are provided by way of illustration only and should not be construed to limit the claims attached hereto. Those skilled in the art will readily recognize various modifications and changes that may be made without following the example embodiments and applications illustrated and described herein, and without departing from the spirit and scope of the following claims.References

[0094] The following publications are incorporated herein by reference in their entirety

[0095] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. NIPS, 2012.

[0096] Rachel Huang, Jonathan Pedoeem, and Cuixian Chen. YOLO-LITE: a real-time object detection algorithm optimized for non-GPU computers. In 2018 IEEE international conference on big data (big data), pages 2503-2510. IEEE, 2018.

[0097] Luciano Floridi and Massimo Chiriatti. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681-694, 2020.Docket # 047162-5382-OOWO

[0098] Ekim Yurtsever, Jacob Lambert, Alexander Carballo, and Kazuya Takeda. A survey of autonomous driving: Common practices and emerging technologies. IEEE Access, 8:58443- 58469, 2020.

[0099] Amir Gholami, Zhewei Yao, Sehoon Kim, Michael W. Mahoney, and Kurt Keutzer. Al and Memory Wall. RiseLab Medium Post, 2021.

[0100] Yu-Hsin Chen, Tushar Krishna, Joel S Emer, and Vivienne Sze. Eyeriss: An energyefficient reconfigurable accelerator for deep convolutional neural networks. JSSCC, 2016.

[0101] Yakun Sophia Shao, Jason Clemons, Rangharajan Venkatesan, Brian Zimmer, Matthew Fojtik, Nan Jiang, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, et al. Simba: Scaling deep-learninginference with multi-chip-module-based architecture. In Proceedings of the 52nd Annual IEEE / ACM International Symposium on Microarchitecture, pages 14-27, 2019.

[0102] Celebras. 18 Gigabytes of on-chip memory, https: / / www.cerebras.net / glossary / memory- hierarchy / , 2020.

[0103] Seyab Khan and Said Hamdioui. Trends and challenges of SRAM reliability in the nanoscale era. In 5th International Conference on Design & Technology of Integrated Systems in Nanoscale Era, pages 1-6. IEEE, 2010.

[0104] Yun Long, Xueyuan She, and Saibal Mukhopadhyay. Design of reliable DNN accelerator with un-reliable ReRAM. In 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1769-1774. IEEE, 2019.

[0105] Dayane Reis, Michael Niemier, and X Sharon Hu. Computing in memory with FeFETs. In Proceedings of the international symposium on low power electronics and design, pages 1-6, 2018.

[0106] Ahmet Inci, Mehmet Meric Isgenc, and Diana Marculescu. Deep-NVM++: cross-layer modeling and optimization framework of nonvolatile memories for deep learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 41(10):3426-3437, 2021.Docket # 047162-5382-OOWO

[0107] Sparsh Mittal and Jeffrey S Vetter. A survey of software techniques for using nonvolatile memories for storage and main memory systems. IEEE Transactions on Parallel and Distributed Systems, 27(5): 1537— 1550, 2015.

[0108] Shimeng Yu, Hongwu Jiang, Shanshi Huang, Xiaochen Peng, and Anni Lu. Compute-inmemory chips for deep learning: Recent trends and prospects. IEEE circuits and systems magazine, 21(3)31-56, 2021.

[0109] Xiaochen Peng, Shanshi Huang, Hongwu Jiang, Anni Lu, and Shimeng Yu. DNN+ NeuroSim V2. 0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020.

[0110] Ki Chui Chun, Pulkit Jain, Tae-Ho Kim, and Chris H Kim. A 667 MHz logic-compatible embedded DRAM featuring an asymmetric 2T gain cell for high speed on-die caches. IEEE Journal of Solid-State Circuits, 47(2)547-559, 2011.

[0111] Mustafa F Ali, Akhilesh Jaiswal, and Kaushik Roy. In-memory lowcost bit-serial addition using commodity DRAM technology. IEEE Transactions on Circuits and Systems I: Regular Papers, 67(1): 155-165, 2019.

[0112] Ki Chui Chun, Pulkit Jain, Jung Hwa Lee, and Chris H Kim. A 3T gain cell embedded DRAM utilizing preferential boosting for high density and low power on-die caches. IEEE Journal of Solid-State Circuits, 46(6)5495-1505, 2011.

[0113] Motoi Ichihashi, Haruki Toda, Yasuo Itoh, and Koichiro Ishibashi. 0.5 V asymmetric three-Tr. cell (ATC) DRAM using 90nm generic CMOS logic process. In Digest of Technical Papers. 2005 Symposium on VLSI Circuits, 2005., pages 366-369. IEEE, 2005.

[0114] Norman P Jouppi, Doe Hyun Yoon, Matthew Ashcraft, Mark Gottscho, Thomas B Jablin, George Kurian, James Laudon, Sheng Li, Peter Ma, Xiaoyu Ma, et al. Ten lessons from three generations shaped google’ s tpuv4i: Industrial product. In ISCA, 2021.

[0115] Guanpeng Li, Siva Kumar Sastry Hari, and Stephen W Keckler. Understanding error propagation in deep learning neural network (DNN) accelerators and applications. InDocket # 047162-5382-OOWOProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, page 8. ACM, 2017.

[0116] Duy-Thanh Nguyen, Nhut-Minh Ho, Minh-Son Le, Weng-Fai Wong, and Ik-Joon Chang. ZEM: Zero-Cycle Bit-Masking Module for Deep Learning Refresh-Less DRAM. IEEE Access, 9:93723-93733, 2021.

[0117] Amit Kazimirsky, Adam Teman, Noa Edri, and Alexander Fish. A 0.65-V, 500-MHz integrated dynamic and static RAM for error tolerant applications. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 25(9):2411-2418, 2017.

[0118] Woong Choi, Gyuseong Kang, and Jongsun Park. A refresh-less eDRAM macro with embedded voltage reference and selective read for an area and power efficient Viterbi decoder. IEEE Journal of Solid-State Circuits, 50(10):2451-2462, 2015.

[0119] Robert Giterman, Alexander Fish, Narkis Geuli, Elad Mentovich, Andreas Burg, and Adam Teman. An 800-MHz Mixed- VT 4T IFGC Embedded DRAM in 28-nm CMOS Bulk Process for Approximate Storage Applications. IEEE Journal of Solid-State Circuits, 53(7):2136- 2148, 2018.

[0120] Robert Giterman, Amir Shalom, Andreas Burg, Alexander Fish, and Adam Teman. A 1- Mbit fully logic-compatible 3T gain-cell embedded DRAM in 16-nm FinFET. IEEE Solid-State Circuits Letters, 3: 110-113, 2020.

[0121] Esteve Amat, Ramon Canal, and Antonio Rubio. Modem gain-cell memories in advanced technologies. In 2018 IEEE 24th International Symposium on On-Line Testing And Robust System Design (IOLTS), pages 65-68. IEEE, 2018.

[0122] Xinfei Guo, Vaibhav Verma, Patricia Gonzalez-Guerrero, Sergiu Mosanu, and Mircea R Stan. Back to the future: Digital circuit design in the finfet era. Journal of Low Power Electronics, 13(3):338— 355, 2017.

[0123] Jungwook Choi, Zhuo Wang, Swagath Venkataram ani, Pierce LJen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv: 1805.06085, 2018.Docket # 047162-5382-OOWO

[0124] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning fdters for efficient convnets. arXiv preprint arXiv: 1608.08710, 2016.

[0125] Morteza Nabavi and Manoj Sachdev. A 290-mV, 3.34-MHz, 6T SRAM with pMOS access transistors and boosted wordline in 65-nm CMOS technology. IEEE Journal of Solid- State Circuits, 53(2):656-667, 2017.

[0126] Seungjae Baek, Sangyeun Cho, and Rami Melhem. Refresh now and then. IEEE Transactions on Computers, 63(12):3114-3126, 2013.

[0127] Yann LeCun et al. LeNet-5, convolutional neural networks. URL: http: / / yann.lecun.com / exdb / lenet, 20(5): 14, 2015.

[0128] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.

[0129] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.

[0130] Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. I- bert: Integer-only bert quantization. In International conference on machine learning, pages 5506-5518. PMLR, 2021.

[0131] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv: 1804.07461, 2018.

[0132] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223-2232, 2017.Docket # 047162-5382-OOWO

[0133] Haotao Wang, Shupeng Gui, Haichuan Yang, Ji Liu, and Zhangyang Wang. Gan slimming: All-in-one gan compression by a unified optimization framework. In European Conference on Computer Vision, pages 54-73. Springer, 2020.

[0134] Yoongu Kim, Ross Daly, Jeremie Kim, Chris Fallin, Ji Hye Lee, Donghyuk Lee, Chris Wilkerson, Konrad Lai, and Onur Mutlu. Flipping bits in memory without accessing them: An experimental study of DRAM disturbance errors. ACM STGARCH Computer Architecture News, 42(3):361-372, 2014.

[0135] Ananda Samajdar, Yuhao Zhu, Paul Whatmough, Matthew Mattina, and Tushar Krishna. Scale-sim: Systolic cnn accelerator simulator. arXiv, 2018.

[0136] Kartik Prabhu, Albert Gural, Zainab F Khan, Robert M Radway, Massimo Giordano, Kalhan Koul, Rohan Doshi, John W Kustin, Timothy Liu, Gregorio B Lopes, et al. CHIMERA: A 0.92-TOPS, 2.2-TOPS / W edge Al accelerator with 2-MByte on-chip foundry resistive RAM for efficient training and inference. IEEE Journal of Solid-State Circuits, 57(4): 1013-1026, 2022.

[0137] Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th annual international symposium on computer architecture, pages 1-12, 2017.

[0138] Ik Joon Chang, Debabrata Mohapatra, and Kaushik Roy. A prioritybased 6T / 8T hybrid SRAM architecture for aggressive voltage scaling in video applications. IEEE Transactions on Circuits and Systems for Video Technology, 21(2): 101— 112, 2011.

[0139] Guodong Yin, Yiming Chen, Mufeng Zhou, Wenjun Tang, Mingy en Lee, Zekun Yang, Tianyu Liao, Xirui Du, Vijaykrishnan Narayanan, Huazhong Yang, et al. Cramming More Weight Data Onto Compute-in-Memory Macros for High Task-Level Energy Efficiency Using Custom ROM With 3984-kb / mm2 Density in 65-nm CMOS. IEEE Journal of Solid-State Circuits, 2023.

[0140] Yunji Chen, Tao Luo, Shaoli Liu, Shijin Zhang, Liqiang He, Jia Wang, Ling Li, Tianshi Chen, Zhiwei Xu, Ninghui Sun, et al. Dadiannao: A machine-learning supercomputer. In 2014Docket # 047162-5382-OOWO47th Annual IEEE / ACM International Symposium on Microarchitecture, pages 609-622. IEEE, 2014.

[0141] Fengbin Tu, Weiwei Wu, Shouyi Yin, Leibo Liu, and Shaojun Wei. RANA: Towards efficient neural acceleration with refresh-optimized embedded DRAM. In 2018 ACM / IEEE 45th Annual International Symposium on Computer Architecture (ISCA), pages 340-352. IEEE, 2018.

[0142] Taegeun Yoo, Hyunjoon Kim, Qian Chen, Tony Tae-Hyoung Kim, and Bongjin Kim. A logic compatible 4T dual embedded DRAM array for in-memory computation of deep neural networks. In 2019 IEEE / ACM International Symposium on Low Power Electronics and Design (ISLPED), pages 1-6. IEEE, 2019.

[0143] Sangwoo Jung, Jaehyun Lee, Huiseong Noh, Jong-Hyeok Yoon, and Jaeha Kung.DualPIM: A Dual-Precision and Low-Power CNN Inference Engine Using SRAM-and eDRAM- based Processing-in-Memory Arrays. In 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 70-73. IEEE, 2022.

[0144] Sangwoo Ha, Sangjin Kim, Donghyeon Han, Soyeon Um, and Hoi-Jun Yoo. A 36.2 dB High SNR and PVT / Leakage-Robust eDRAM Computing-In-Memory Macro With Segmented BL and Reference Cell Array. IEEE Transactions on Circuits and Systems II: Express Briefs, 69(5):2433-2437, 2022.

Claims

Docket # 047162-5382-OOWOCLAIMSWhat is claimed is:

1. An on-chip artificial intelligence memory system, comprising: a mixed-memory cell comprising a static random access memory (SRAM) cell and at least one embedded dynamic random access memory (eDRAM) cell, wherein the mixed-memory cell is configured to map a control bit of encoded data to the SRAM cell and to map remaining bits of the encoded data to the at least one eDRAM cell; an enhancement encoder decoder module configured to receive and encode data for storage in the mixed-memory cell; and a refresh controller configured to refresh at least one data bit stored in the at least one eDRAM cell .

2. The system of claim 1, wherein the enhancement encoder-decoder module comprises a one-enhancement encoder-decoder module.

3. The system of claim 1, wherein the enhancement encoder-decoder module comprises an inverter and at least one XOR gate, and is configured to invert the at least one data bit stored in the at least one eDRAM cell when the control bit stored in the SRAM cell is bit- 0.

4. The system of claim 1, wherein the SRAM cell comprises a six transistor SRAM cell and the at least one eDRAM cell comprises at least one two transistor eDRAM cell.

5. The system of claim 4, wherein the at least one two transistor eDRAM cells comprise seven two-transistor eDRAM cells.

6. The system of claim 4, wherein the mixed-memory cell is configured to map the control bit to the six transistor SRAM cell and seven less significant bits to the respective two transistor eDRAM cells.Docket # 047162-5382-OOWO7. The system of claim 4, wherein the at least one two transistor eDRAM cell is asymmetric.

8. The system of claim 4, wherein the at least one two transistor eDRAM cell comprises a N-channel Metal-oxide Semiconductor (NMOS) access transistor and a storage node.

9. The system of claim 8, wherein the storage node comprises a capacitor.

10. The system of claim 4, wherein the six transistor SRAM cell comprises a P-channel Metal-oxide Semiconductor (PMOS) access transistor and a storage node.

11. The system of claim 1, further comprising a voltage sense amplifier coupled to the SRAM cell or the eDRAM cell.

Images