Computing-in-memory processor and method for performing hybrid operation for mixed-precision deep neural network operation
The mixed-precision DNN inference processor addresses throughput limitations and ADC overhead in CIM by separating and processing low-bit inliers and high-bit outliers in parallel, enhancing energy efficiency and throughput.
Patent Information
- Application Number
- KR1020230101219
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-12-16
- Filing Date
- 2023-08-02
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2043-08-02
AI Technical Summary
Conventional in-memory computing (CIM) processors face throughput limitations and ADC overhead due to bit serial operations and the need to wait for high-bit outliers, leading to inefficient mixed-precision computation in Deep Neural Networks (DNNs).
A mixed-precision DNN inference processor that separates low-bit inliers and high-bit outliers, performs analog CIM on inliers and digital CIM on outliers in parallel, using a hierarchical cell array structure with local-cell array groups (LAGs) to process operations efficiently.
Improves computation efficiency by preventing throughput limitations, minimizing ADC overhead, and reducing power waste, achieving up to a 76.59% reduction in energy consumption and a 2x increase in throughput.
Smart Images

Figure 112023085362621-PAT00003_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a Computing-In-Memory (CIM) processor, and more particularly, to a CIM processor capable of supporting mixed-precision DNN computation, and a mixed-precision DNN inference processor applying the same. Background Technology
[0002] Deep Neural Networks (DNNs) are a field of machine learning that is currently being utilized in various areas such as image analysis, object recognition, and image segmentation. They generate output by accumulating the multiplication of activations and weights based on Multiplication and Accumulation (MAC) operations.
[0003] Recently, In-Memory Computing (CIM) processors have been attracting attention as next-generation DNN accelerators. In-Memory Computing (CIM) processors are designed to solve the problem of excessive memory power consumption caused by the large amount of data movement during DNN computations, and aim to reduce the amount of memory access by supporting computation functions within memory (References 1, 2).
[0004] These in-memory computing (CIM) processors were able to achieve the highest level of energy efficiency by executing multiple Multiply Accumulate (MAC) operations through a single memory access.
[0005] Meanwhile, to improve computational energy efficiency by lowering bit precision, DNN processors adopt a mixed-precision computation method that classifies data values within each layer into low-precision inliers and high-precision outliers, sets the ratio of inliers to a very high value, and then performs computation.
[0006] Figure 1 is a diagram illustrating the mixed-precision deep neural network of such a general DNN, and illustrates the effect of using mixed-precision operations with a ResNet18 network on the CIFAR-100 dataset.
[0007] Referring to Figure 1, under the above conditions, an input characteristic map is shown in which 97% of the input data is a fixed-point 4-bit (FIX4) inlier and 3% of the input data is a fixed-point 8-bit (FIX8) outlier. It illustrates that when a mixed precision operation is performed by applying an 8-bit weight after setting the ratio of inliers to a very high level, the average input precision can be reduced by 48.5% compared to the precision optimized by hierarchy.
[0008] However, when implementing mixed precision using a conventional in-memory computing (CIM) processor, there is a problem in that it is difficult to obtain the effect exemplified in Fig. 1 due to throughput limitations and ADC overhead.
[0009] This is because conventional CIM processors use bit serial (BS) operations, which calculate the position of a different bit for each clock cycle.
[0010] In other words, conventional CIM processors have a characteristic that if any of the multiple word lines transmitting input data transmit a high-bit outlier, they must wait until the operation of that outlier is finished. Due to this characteristic, the throughput of conventional CIM processors is limited by high-bit outliers.
[0011] In addition, conventional CIM processors are equipped with an Analog-to-Digital Converter (ADC) for each bit line to convert analog data accumulated in bit line units into data, and the ADC is integrated with high bits to calculate inliers, but when outliers are input, only low-bit calculation results are needed, so there was a problem of wasted ADC power.
[0012] These problems are summarized and illustrated in Figure 2.
[0013] Figure 2 is a diagram illustrating problems that occur during mixed-precision calculations using a conventional analog CIM processor, illustrating two problems (① Throughput Limitation) and limited accumulation of outliers (② Limited Accumulation of Outliers) that occur during mixed-precision calculations using a conventional analog CIM processor.
[0014] Referring to Figure 2, it can be seen that conventional analog CIM structures cannot take advantage of the energy-efficient computation of mixed precision due to the heterogeneous nature of the cumulative computation resulting from the use of mixed precision. Prior art literature
[0015] Reference 1: H. Jia et al., "15.1 A Programmable Neural-Network Inference Accelerator Based on Scalable In-Memory Computing," 2021 IEEE International Solid-State Circuits Conference (ISSCC), 2021, pp. 236-238Reference 2: J. Yue et al., "15.2 A 2.75-to-75.9TOPS / W Computing-in-Memory NN Processor Supporting Set-Associate Block-Wise Zero Skipping and Ping-Pong CIM with Simultaneous Computation and Weight Updating," 2021 IEEE International Solid-State Circuits Conference (ISSCC), 2021, pp. 238-240 The problem to be solved
[0016] Accordingly, the present invention aims to provide an in-memory computing processor (CIM processor) and a method thereof that can improve the efficiency of mixed-precision computation of a DNN by preventing throughput limitations caused by outliers and minimizing ADC overhead when applying mixed-precision computation.
[0017] In addition, the present invention aims to provide an in-memory computing processor (CIM processor) and a method thereof that can improve the efficiency of mixed-precision computation in a DNN by processing the accumulation of high-bit outliers that are sparse and irregularly distributed during mixed-precision computation using digital CIM computation instead of conventional analog CIM computation, and by processing analog CIM computation and digital CIM computation in parallel.
[0018] In addition, the present invention aims to provide an in-memory computing processor (CIM processor) and a method thereof that can reduce power waste for operating an ADC and improve computation speed by processing the processing results of digital CIM operations in a digital peripheral circuit rather than an ADC.
[0019] In addition, the present invention aims to provide an in-memory computing processor (CIM processor) and a method thereof that can improve digital CIM computation efficiency by adopting a hierarchical cell array structure comprising a plurality of local-cell array groups (LAGs) and processing each of the LAGs in parallel.
[0020] In addition, the present invention aims to provide an in-memory computing processor (CIM processor) and a method thereof that improves digital CIM throughput per unit time by performing two multiplications and one addition in a single cycle, by pre-charging the value of a bit line as a logical AND of first and second input characteristic values, activating a first word line applied to an arbitrary first LAG as the first input characteristic value, and activating a second word line applied to an arbitrary second LAG different from the first LAG as the second input characteristic value. means of solving the problem
[0021] To achieve the above-mentioned objective, the mixed-precision DNN inference processor provided in the present invention comprises: an input memory that stores data input for mixed-precision computation, wherein low-bit inliers and high-bit outliers are stored separately; a plurality of CIM operators that perform Computing In Memory (CIM) computations, wherein the inliers are subjected to analog CIM computation and the outliers are subjected to digital CIM computation, and then summed to produce an output; an aggregator that aggregates the computation results of each of the plurality of CIM operators; an output memory that stores the computation results of the aggregator; and a precision encoding unit that converts the computation results stored in the output memory into mixed-precision inputs.
[0022] Preferably, the CIM operator may include a plurality of memory cells storing weights, a cell array operation unit that performs at least one of bit-unit analog CIM operation for an inlier existing within a predetermined operation cycle and bit-unit digital CIM operation for an outlier existing within the operation cycle, and an accumulation operation unit that accumulates the results of the analog CIM operation and the digital CIM operation.
[0023] Preferably, the cell array operation unit can process the analog CIM operation and the digital CIM operation in parallel by sharing a predetermined number of weights among those referenced for the analog CIM operation during the digital CIM operation.
[0024] Preferably, the cell array operation unit adopts a hierarchical cell array structure comprising a plurality of local-cell array groups (LAGs) composed of a predetermined number of memory cells, and can process the operation of each LAG in parallel.
[0025] Preferably, the cell array operation unit, during the digital CIM operation, pre-charges the value of a bit line applied to an arbitrary first LAG as a logical AND of arbitrary first and second input characteristic values (I1, I2), then activates the first word line applied to the first LAG as the first input characteristic value (I1) to generate a first logical AND result and a first logical OR result, and can accumulate the input and weighted product using the first logical AND result and the first logical OR result.
[0026] Preferably, when performing the digital CIM operation, the cell array operation unit pre-charges the value of a bit line applied to an arbitrary second LAG different from the first LAG as a logical AND of arbitrary third and fourth input characteristic values (I3, I4), then activates the second word line applied to the second LAG as the third input characteristic value (I3) to generate a second logical AND result and a second logical OR result, and accumulates the input and weighted product using the second logical AND result and the second logical OR result to produce a digital CIM operation result of the second LAG that is different from the digital CIM operation result of the first LAG.
[0027] Preferably, the memory cell may include a 6T SRAM cell operating by a bit-line, a bit-line bar, and a word line; a plurality of transistors coupled to an operation word line (CWL) and an operation bit line (CBL); and a cell capacitor (C) connected between the operation bit line (CBL) and the plurality of transistors.
[0028] Meanwhile, to achieve the above objective, the mixed-precision DNN inference method provided by the present invention is characterized by comprising: a data storage step in which the DNN inference processor stores data input for mixed-precision computation, and separates and stores low-bit inliers and high-bit outliers; a CIM computation step in which each of the plurality of CIM computation units performs CIM computation using the stored data, wherein the inliers are computed using an analog CIM method and the outliers are computed using a digital CIM method, and then summed to produce an output; an aggregation step in which the computation results of each of the plurality of CIM computation units are aggregated; and a precision encoding step in which the computation result of the aggregation step is converted into a mixed-precision input.
[0029] Preferably, the CIM operation step may include: an analog CIM operation step that performs bit-based accumulation of inliers located within a predetermined operation cycle and outputs the result after ADC conversion; a digital CIM operation step that performs bit-based operation on outliers located within the operation cycle and outputs the result after accumulation by logical operation; and an accumulation operation step that accumulates the result of the analog CIM operation and the result of the digital CIM operation.
[0030] Preferably, the digital CIM operation step can be processed in parallel with the analog CIM operation step by sharing a predetermined number of weights among those referenced in the analog CIM operation step, which are stored in a memory cell constituting the CIM operation unit.
[0031] Preferably, the CIM operation step can be processed in parallel in units of a plurality of local cell array groups (LAGs) composed of a predetermined number of memory cells.
[0032] Preferably, the digital CIM operation step comprises: a first pre-charge step for pre-charging the value of a bit line applied to an arbitrary first LAG as the logical AND of arbitrary first and second input characteristic values (I1, I2); a first activation step for activating a first word line applied to the first LAG with either the first or second input characteristic value (I1 or I2); a first logical AND operation step for generating a first logical AND, which is the product of the input and weights, in a memory cell connected to the activated word line; and It may include a first logical OR operation step that accumulates the first logical AND result in bitline units.
[0033] Preferably, the digital CIM operation step further comprises: a second pre-charge step for pre-charging the value of a bit line applied to any second LAG different from the first LAG as a logical AND of any third and fourth input characteristic values (I3, I4); a second activation step for activating a second word line applied to the second LAG as either the third or fourth input characteristic value (I3 or I4); a second logical AND operation step for generating a second logical AND, which is a product of the input and weights, in a memory cell connected to the activated word line; and a second logical OR operation step for accumulating the second logical AND result in bit line units, thereby enabling the output of a plurality of digital CIM operation results within a single operation cycle for performing the analog CIM operation step. Effects of the invention
[0034] As such, the in-memory computing processor and method provided in the present invention have the effect of improving the efficiency of mixed-precision computation of a DNN by preventing throughput limitations caused by outliers and minimizing ADC overhead when applying mixed-precision computation.
[0035] In addition, the in-memory computing processor and method provided in the present invention have the effect of improving the efficiency of mixed-precision computation of a DNN by processing the accumulation of sparse and irregularly distributed high-bit outliers during mixed-precision computation using digital CIM computation instead of conventional analog CIM computation, and by processing analog CIM computation and digital CIM computation in parallel.
[0036] In addition, the in-memory computing processor and method provided in the present invention have the effect of reducing power waste for operating the ADC and improving the computation speed by processing the processing result of the digital CIM operation in a digital peripheral circuit rather than the ADC.
[0037] In addition, the in-memory computing processor and method provided by the present invention utilize a hierarchical cell array structure comprising a plurality of local-cell array groups (LAGs) and have the effect of improving digital CIM computation efficiency by processing each of the LAGs in parallel.
[0038] In addition, the in-memory computing processor and method provided in the present invention have the effect of improving CIM throughput per unit time by performing two multiplications and one addition in a single cycle, by pre-charging the value of a bit line as a logical AND of a first and second input characteristic values, activating a first word line applied to an arbitrary first LAG as the first input characteristic value, and activating a second word line applied to an arbitrary second LAG different from the first LAG as the second input characteristic value. Brief explanation of the drawing
[0039] Figure 1 is a diagram illustrating a mixed-precision deep neural network of a general DNN. Figure 2 is a diagram illustrating the problems that occur during mixed-precision calculations using a conventional analog CIM processor. FIG. 3 is a schematic block diagram of a DNN inference processor according to one embodiment of the present invention. FIG. 4 is a schematic block diagram of a CIM calculator according to one embodiment of the present invention. FIG. 5 is a diagram illustrating an overview of mixed precision calculation using a CIM calculator according to one embodiment of the present invention. FIG. 6 is a timing diagram illustrating the timing for processing analog and digital operations in parallel during mixed precision calculation using a CIM calculator according to one embodiment of the present invention. FIG. 7 is a block diagram illustrating a detailed configuration of a cell array operation unit according to one embodiment of the present invention. FIG. 8 is a diagram illustrating an analog calculation process using a CIM calculator according to an embodiment of the present invention. FIG. 9 is a circular graph comparing the amount of energy consumed during analog and digital calculations using a CIM calculator according to one embodiment of the present invention. FIG. 10 is a diagram illustrating a digital computation process using a CIM calculator according to an embodiment of the present invention. FIG. 11 is a table summarizing bit-line values determined according to input data values during digital operation using a CIM calculator according to one embodiment of the present invention. FIG. 12 is a diagram illustrating the reduction of pre-charge power and improvement of throughput during digital computation using a CIM computer according to one embodiment of the present invention. FIG. 13 is a diagram showing the design of a mixed-precision DNN inference processor according to one embodiment of the present invention, and a diagram showing the results of DNN operations using the same and the performance of the processor in a table. FIGS. 14 to 16 are process flowcharts illustrating a mixed precision DNN inference method according to an embodiment of the present invention. Specific details for implementing the invention
[0040] Embodiments of the present invention are described below with reference to the attached drawings. The description is provided in detail to enable those skilled in the art to easily practice the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Meanwhile, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification have been given similar reference numerals. Furthermore, explanations of parts that can be easily understood by those skilled in the art even without detailed description have been omitted.
[0041] Throughout the specification and claims, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.
[0042] Hereinafter, in order to provide a detailed explanation so that a person skilled in the art to which the present invention pertains can easily implement the present invention, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.
[0043] FIG. 3 is a schematic block diagram of a DNN inference processor according to an embodiment of the present invention. Referring to FIG. 3, a DNN inference processor (100) according to an embodiment of the present invention includes an input memory (110), a plurality of CIM arithmetic units (120) (M3-CIM Core #1 to #4), an aggregator (130), an output memory (140), a precision encoding unit (150), and a top controller (160), and these are connected by a network-on-chip in the form of a 2D mesh.
[0044] The input memory (110) stores data input for mixed precision operations, and stores low-bit inliers and high-bit outliers separately. In particular, the input memory (110) stores the lower bit values of the high-bit outliers together with the low-bit inliers, and stores the upper bit values of the high-bit outliers in a separate area. For example, if the inlier is 4 bits and the outlier is 8 bits, the input memory (110) stores the lower 4 bit values of the outlier together with the 4-bit inliers, and stores the upper 4 bit values of the outlier in a separate area.
[0045] In this case, the lower bit values of inliers and outliers have a dense characteristic with a high frequency of occurrence, while the upper bit values of outliers have a sparse characteristic with a low frequency of occurrence (i.e., a sparse and irregularly distributed characteristic). Therefore, the lower bit values of inliers and outliers can be classified as a Dense Accumulation Slice (DAS), and the upper bit values of outliers can be classified as a Sparse Accumulation Slice (SAS) and stored separately.
[0046] To this end, the input memory (110) may be classified into a DAS memory (111) for storing DAS and a SAS memory (112) for storing SAS, and the SAS memory (112) may be implemented to have a relatively very small capacity compared to the DAS memory (111) in order to store sparse and irregularly distributed SAS. In the example of FIG. 3, an example is illustrated in which the input memory (110) is composed of a DAS memory (111) with a capacity of 32KB and a SAS memory (112) with a capacity of 1KB.
[0047] The CIM operator (120) performs Computing In Memory (CIM) operations, wherein the inlier performs analog CIM operations and the outlier performs digital CIM operations, and then sums and outputs the results, and multiple CIM operators may be included in a single DNN inference processor (100). In the example of FIG. 3, an example is illustrated in which four CIM operators (M3-CIM Core #1~#4) (120) are included in a single DNN inference processor (100).
[0048] A detailed description of the configuration and operation of the CIM calculator (120) will be explained with reference to FIGS. 4 to 12.
[0049] The aggregator (130) aggregates the results of each of the multiple CIM arithmetic units (120) and stores the result in the output memory (140).
[0050] The output memory (140) stores the operation result of the summer (130). In the example of FIG. 3, an example is illustrated in which the DNN inference processor (100) includes an output memory (140) with a capacity of 4KB.
[0051] The precision encoding unit (150) converts the operation result stored in the output memory into a mixed precision input. To this end, the precision encoding unit (150) classifies the operation result into inliers and outliers according to their size, determines the precision based on this, and stores it in memory.
[0052] The top controller (TOP Ctrl.) (160) controls the overall operation of the DNN inference processor (100), in particular, manages the communication of each component (i.e., input memory (110), multiple CIM operators (120), summator (130), output memory (140), and precision encoding unit (150)), and performs general processing required for DNN inference, such as batch normalization and activation function operations.
[0053] FIG. 4 is a schematic block diagram of a CIM operator according to an embodiment of the present invention, illustrating the configuration of each of a plurality of CIM operators (120) included in the DNN inference processor (100) illustrated in FIG. 3. Referring to FIG. 3 and FIG. 4, the CIM operator (120) includes a DAS buffer (121), an analog decoder (122), a SAS buffer (123), a digital decoder (124), a cell array operation unit (M3-CIM cell array) (200), and an accumulation circuit (also known as an accumulation operator) (125).
[0054] At this time, the cell array operation unit (200) includes a plurality of memory cells that store weights and performs CIM operations in units of a predetermined operation cycle, wherein at least one of bit-unit analog CIM operations for inliers existing within the operation cycle and bit-unit digital CIM operations for outliers existing within the operation cycle is performed. In particular, the cell array operation unit (200) performs analog CIM operations on DAS bits stored in DAS memory (111) and performs digital CIM operations on SAS bits stored in SAS memory (112).
[0055] To this end, the DAS bit is transmitted to the cell array operation unit (M3-CIM cell array) (200) via the DAS buffer (121) and the analog decoder (122), and the SAS bit is transmitted to the cell array operation unit (M3-CIM cell array) (200) via the SAS buffer (123) and the digital decoder (124).
[0056] Meanwhile, the results of the analog CIM operation and the digital CIM operation are summed in the peripheral circuit (Accumulation) (also known as the accumulation unit) (125). At this time, the analog CIM operation is converted into a digital signal and output through the ADC included in the peripheral circuit (Accumulation) (also known as the accumulation unit) (125) by adopting a standard CIM operation method, and the digital CIM operation can be accumulated by a logical OR operation in bitline units.
[0057] FIG. 5 is a diagram for explaining an overview of mixed precision calculation using a CIM calculator according to an embodiment of the present invention, illustrating how a cell array calculation unit (200) included in the CIM calculator (120) processes mixed precision calculation.
[0058] Referring to FIGS. 4 and FIG. 5, the cell array operation unit (200) of the present invention independently processes DAS bits and SAS bits by utilizing heterogeneous data paths. That is, analog CIM operations are performed on DAS bits input to the cell array operation unit (200) from the left side of the drawing through a transmission line formed via a DAS buffer (121) and an analog decoder (122), and digital CIM operations are performed on SAS bits input to the cell array operation unit (200) from the right side of the drawing through a transmission line formed via a SAS buffer (123) and a digital decoder (124).
[0059] As a result, the cell array operation unit (200) of the present invention can perform the sparse and irregularly distributed cumulative operation appearing in the top bit (SAS bit) of the outlier using a digital CIM method, rather than using the existing analog CIM method.
[0060] Meanwhile, the cell array operation unit (200) of the present invention, as illustrated in FIG. 5, adopts a method of activating multiple word-lines and using an ADC similar to a conventional CIM processor for high-density accumulation of DAS bits, and performs low-density accumulation of SAS bits by replacing the power-consuming ADC part with a digital peripheral circuit. It is well known that such digital CIM operations are highly energy-efficient in irregular and sparse accumulation operations.
[0061] This is intended to resolve conventional problems arising from distinctly different accumulation characteristics in the DAS and SAS stages during mixed-precision DNN operations. Specifically, for operations in the DAS stage, where a large amount of accumulation is always guaranteed, energy efficiency gains are achieved by adopting conventional analog CIM operations to offset large ADC power. Conversely, for operations in the SAS stage, where an accumulation large enough to offset ADC power is not guaranteed, energy efficiency is improved by adopting digital CIM operations and using digital logic instead of ADC operations.
[0062] Additionally, the cell array operation unit (200) can process the analog CIM operation and the digital CIM operation in parallel by sharing a predetermined number of weights among those referenced for the analog CIM operation during the digital CIM operation.
[0063] FIG. 6 is a timing diagram (A) illustrating the timing for processing analog and digital operations in parallel during mixed precision calculation using a CIM calculator according to one embodiment of the present invention. Referring to FIG. 6, it is shown that the CIM calculator of the present invention can increase the throughput by twofold by simultaneously processing analog CIM and digital CIM in parallel.
[0064] Meanwhile, this structure has strengths in two major aspects.
[0065] First, by performing operations on the SAS unit as a digital CIM, throughput can be increased without unnecessary memory requirements. That is, since each cell constituting the CIM calculator (120) is capable of both analog CIM operations and digital CIM operations, it does not require additional memory, and throughput can be increased without redundant weight memory between the analog CIM and the digital CIM.
[0066] Second, during low-density accumulation operations, by utilizing the accumulation operation of digital peripheral circuits without an ADC, the energy consumed by the ADC can be reduced, thereby increasing the overall energy efficiency of the processor.
[0067] In addition, the cell array operation unit (200) of the present invention adopts a hierarchical cell array structure comprising a plurality of local-cell array groups (LAG) composed of a predetermined number of memory cells, and processes the operation of each of the LAGs in parallel.
[0068] This is for operating the digital CIM with high energy efficiency, and an example of such a cell array operation unit (200) structure is illustrated in FIG. 7.
[0069] FIG. 7 is a block diagram illustrating a detailed configuration of a cell array operation unit according to an embodiment of the present invention. Referring to FIG. 7, the cell array operation unit (200) according to an embodiment of the present invention consists of 1,152 × 256 cells, and 288 cells are grouped to form 1,024 local array groups (LAG) (210). Additionally, each local array group (LAG) (210) is composed of 18 local cell arrays (LCA) (211) each composed of 16 cells, and the LCA (211) is composed of 16 cells, a local pre-charger, a local bit-line (local BL, LBL), and a local bit-line bar (local BLB, LBLB).
[0070] Additionally, each local array group (LAG) (210) is connected to a digital peripheral circuit (10) and an ADC (20) on a bitline basis to accumulate digital CIM operation results and analog CIM operation results on a bitline basis.
[0071] Each cell (300) is configured to include a 6T SRAM cell (310) that operates via a bit-line (BL), a bit-line bar (BLB), and a word line (WL), a plurality of transistors (320) that operate via a word line (CWL) and a bit line (CBL), and a cell capacitor (C) (330) connected between the bit line (CBL) and the plurality of transistors (320). At this time, the cell capacitor (C) (330) is configured as a custom-layouted Metal-Oxide-Metal (MOM) capacitor with a capacitance of 1.14 fF and is implemented by stacking on top of each cell.
[0072] Typically, an SRAM cell stores data using six transistors (i.e., a 6T SRAM cell (310)), but the memory cell (300) of the present invention enables in-memory computing by adding a plurality of transistors (320) and a cell capacitor (C) (330). In particular, the memory cell (300) is characterized by being capable of parallel operation by utilizing not only the cell capacitor (C) (330) but also six transistors (i.e., a 6T SRAM cell (310)).
[0073] Additionally, in the example of FIG. 7, 1,152 cells are connected to each BL, and an example is illustrated in which this is divided into four local cell array groups (LAG) (210). In this case, each of the four LAGs (210) has an independent peripheral circuit, and they can operate in parallel.
[0074] FIG. 8 is a diagram for explaining an analog operation process using a CIM operator according to an embodiment of the present invention, where FIG. 8 (a) shows a data path of a cell array operation unit (200) used for DAS bit operation, and FIG. 8 (b) to (d) illustrate the operation process.
[0075] Referring to FIGS. 7 and 8, to perform analog CIM operations, the cell array operation unit (200) uses a CWL, a cell capacitor (C) (330), a CBL, and an 8-bit ADC (20). In this case, the cell array operation unit (200) has 1,152 cells forming a column, and each cell constituting the column is connected to the CBL through the cell capacitor (C) (330). Meanwhile, for analog CIM operations, the CWL is activated with a DAS bit value, and the product of each DAS bit and the weight within the cell is bootstrapped into the CBL to reflect the operation result. Additionally, each bootstrapped product is accumulated within the CBL. The detailed operation process can be seen in FIGS. 8 (b) to (d). Referring to FIG. 8(b), initially, the CBL is initialized to 0 via an initialization signal (reset, RST); referring to FIG. 8(c), the CWL driver in the analog decoder is activated with the DAS bit value (1001); and referring to FIG. 8(d), finally, the analog value accumulated in the CBL is converted into a digital value through an 8-bit ADC operation. At this time, since the operation of the ADC (20) takes 9 clock cycles, the CWL driver must process the next DAS bit after waiting 9 cycles.
[0076] FIG. 9 is a pie chart comparing the amount of energy consumed during analog and digital operations using a CIM calculator according to one embodiment of the present invention, showing the energy configuration for a 1-bit input operation in the CIFAR-100 dataset.
[0077] Referring to Figure 9, it can be seen that the analog CIM and digital CIM consume 81% and 19% of the energy of the CIM computer, respectively, and that a 76.59% reduction in energy can be achieved by replacing the analog CIM operation with the digital CIM at the SAS stage. In addition, up to a 2x increase in throughput can be achieved by operating the analog CIM and digital CIM in parallel. Therefore, the CIM computer can achieve 55.46% higher energy efficiency than when performing calculations using only the analog CIM.
[0078] As described above, the present invention secures the possibility of increasing throughput by performing operations on DAS and SAS bits in the analog CIM and digital CIM, respectively. However, the operation time of the analog CIM is proportional to the number of DAS bits and the ADC operation time per bit. Although the operation time of the digital CIM must be shorter than that of the analog CIM to increase the throughput of the CIM operator, existing digital CIM structures find it difficult to meet the throughput requirements of the CIM operator. This is because the operation time of existing digital CIM structures is long due to the limited parallelism of the digital CIM structures. In other words, existing digital CIM structures limit parallelism because they perform only one multiplication or one addition per cycle, which leads to a problem of long operation times.
[0079] Accordingly, the present invention satisfies the throughput requirements of a CIM operator by performing two multiplications and one addition in a single cycle during digital CIM operations. To this end, the CIM operator of the present invention can free-occupy a local BL (LBL) using the result of a logical AND of two input feature maps, activate the value of a word-line (WL) as an input feature value to generate a first logical AND and a first logical OR result, and accumulate the input and weighted product using the first logical AND result and the first logical OR result.
[0080] This digital CIM operation structure is exemplified in Fig. 10.
[0081] FIG. 10 is a diagram illustrating a digital operation process using a CIM calculator according to an embodiment of the present invention, where FIG. 10 (a) illustrates the structure of an arbitrary local array group (LAG#0) (210), and FIG. 10 (b) illustrates an example of performing digital operations on two of the local cell arrays (211a, 211b) included in the LAG#0 (210).
[0082] Referring to FIG. 10(a), the local array group (LAG#0) (210) includes 18 local cell arrays (LCA) connected to GBL and GBLB, and each operation result is accumulated by a logic circuit (30) connected to the bottom. In this way, the CIM operator of the present invention can improve energy efficiency by accumulating operation results through a logic circuit (i.e., an adder) (30) implemented as a digital circuit instead of an ADC, which consumes a large amount of power during digital CIM operations.
[0083] The logic circuit (30) for this purpose may include a first adder (2 Cell-Adder) (31) that accumulates the operation results of LAG units as exemplified in FIG. 10, and a second adder (Full-Adder) (32) that accumulates the operation results of each LAG.
[0084] Referring to FIG. 10(b), it can be seen that for digital CIM operations, the CIM operator of the present invention frees up the local BL (LBL) of LCA#0 (211a) using the result of the logical AND of the 0th and 1st input characteristic maps (I0, I1), activates the value of the word-line (WL1) of LCA#0 (211a) using the 1st input characteristic map (I1), frees up the local BL (LBL) of LCA#1 (211b) using the result of the logical AND of the 0' and 1' input characteristic maps (I'0, I'1), and activates the value of the word-line (WL0) of LCA#1 (211b) using the 0' input characteristic map (I'0).
[0085] As such, the CIM computing unit of the present invention utilizes a hierarchical cell array structure including a plurality of LAGs, and by processing each of the LAGs in parallel, it can improve digital CIM computing efficiency.
[0086] That is, in the example of FIG. 10 (b), a plurality of logical ANDs can be obtained by processing each of the plurality of LAGs (211a and 211b) in parallel in one cycle, and a logical OR can be obtained by accumulating the two values.
[0087] In this way, the present invention utilizes the logical AND and logical OR results, which are in-memory operation results, to use fewer gates during operation and can use them as inputs for a full adder, thereby improving energy efficiency.
[0088] Meanwhile, the operation of the digital CIM according to one embodiment of the present invention is largely composed of four steps.
[0089] First, the local BL (LBL) and local BLB (LBLB) values are pre-charged with the logical AND of the input and the supply voltage, respectively.
[0090] Second, the word-line value is activated as the value of the first input characteristic bit. A bitwise logical AND of the input is performed between the LBL and the weights within the cell, and for this purpose, the maximum voltage of the WL is set to 70% of the supply voltage. The reason for setting the maximum voltage of the WL to 70% of the supply voltage is to ensure that the value stored in the cell, which consists of six transistors, does not change.
[0091] Third, the word-line value at a different location is activated as the second input characteristic bit value. Through this, the global BL (GBL) driver converts the GBL and GBLB values into LBL and LBLB values. At this time, the LBL and LBLB values generated during each operation process are exemplified in FIG. 11. FIG. 11 is a table summarizing the bit-line values determined according to the input data value during digital operation using a CIM operator according to an embodiment of the present invention, and the LBL / LBLB / GLB / GLBL values are exemplified in the table when the zero and first input characteristic maps (I0, I1) are (0,0), (0,1), (1,0), and (1,1).
[0092] Finally, a digital peripheral circuit consisting of an adder ('30' in Fig. 10) completes the accumulation process.
[0093] At this time, the operations of each of the above four steps are pipelined, thereby minimizing the reduction in throughput of each step.
[0094] By doing so, the present invention makes it possible to obtain multiple digital CIM operation results in one cycle from a single LAG.
[0095] FIG. 12 is a diagram illustrating the reduction of pre-charge power and improvement of throughput during digital computation using a CIM computer according to an embodiment of the present invention. When performing digital computation using the CIM computer of the present invention, there is an effect of reducing the pre-charge power of the BL while increasing throughput.
[0096] Referring to FIG. 12, when performing digital operations using the CIM computer of the present invention, BL pre-charge power could be reduced by 94.5% through a hierarchical cell array, and pre-charge power could be reduced by up to 99.7% by pre-charging the BL value as the logical AND value of the input. In addition, the operation time of the digital CIM could be hidden within the operation time of the analog CIM, thereby improving the throughput of the entire CIM computer by 41.27% on the CIFAR-100 dataset.
[0097] In other words, while existing analog CIM operations had to be performed according to the outlier bit length, the present invention is designed so that digital CIM and analog CIM operate in parallel, thereby achieving the effect of performing operations according to the inlier bit length. In addition, existing digital CIMs have too low throughput, which can result in processing times being longer than the inlier bit processing times of analog CIMs; to solve this, the present invention doubles the throughput of the digital CIM, thereby achieving an overall throughput improvement of 41.27% as shown in the example above.
[0098] In addition, by referring to the circular graph in Fig. 12, the detailed power consumption of the digital CIM can be confirmed. That is, by referring to the circular graph in Fig. 12, it can be seen that during the digital CIM, the power consumption of the GBL driver of the CIM computing device of the present invention is 39%, the power consumption of the decoder (Dedoder) is 5%, the power consumption of the precharge is 33%, and the power consumption of the cell power is 23%.
[0099] FIG. 13 is a diagram showing the design of a mixed-precision DNN inference processor according to an embodiment of the present invention, the results of DNN computation using the same, and the performance of the processor in a diagram. Referring to FIG. 13, it can be seen that each CIM operator (120) has 36KB of memory integrated, the analog CIM operates at a clock frequency of 200 MHz and the ADC operates at 20 MHz with a supply voltage of 1V, and the digital CIM operates at a clock frequency of 200 MHz and uses supply voltages of 1V and 0.7V.
[0100] FIGS. 14 to 16 are process flowcharts illustrating a mixed-precision DNN inference method according to an embodiment of the present invention. Referring to FIG. 3 and FIGS. 14 to 16, a DNN inference method using a mixed-precision DNN inference processor according to an embodiment of the present invention is as follows.
[0101] First, in step S100, the DNN inference processor (100) stores the data input for mixed precision operations, and separates and stores low-bit inliers and high-bit outliers in the input memory (110).
[0102] In step S200, each of the plurality of CIM operators (120) performs a CIM operation using the stored data, wherein the inlier is operated on using an analog CIM method and the outlier is operated on using a digital CIM method, and then summed and output. Additionally, step S200 can be processed in parallel in units of a plurality of local-cell array groups (LAG) composed of a predetermined number of memory cells. To this end, each of the plurality of CIM operators (120) may adopt a hierarchical cell array structure that includes a plurality of local-cell array groups (LAG) composed of a predetermined number of memory cells.
[0103] Meanwhile, in step S210, each of the CIM operators (120) accumulates inliers located within a predetermined operation cycle bit by bit, converts them to ADC, and outputs the result.
[0104] In step S220, each of the CIM operators (120) performs bit-wise operations on outliers located within the operation cycle and then accumulates and outputs them through logical operations. At this time, step S220 can be processed in parallel with step S210. To this end, each of the CIM operators (120) may share a predetermined number of weights among those referenced in step S210 (i.e., the analog CIM operation step) that are stored in memory cells constituting the CIM operator (120).
[0105] Additionally, to perform the above step S220 (i.e., digital CIM operation step), in step S221, each of the CIM operators (120) pre-charges the value of a bit line applied to an arbitrary first LAG as a logical AND of arbitrary first and second input characteristic values (I1, I2); in step S222, activates a first word line applied to the first LAG with either the first or second input characteristic value (I1 or I2); in step S223, generates a first logical AND that is a product of the input and weights in a memory cell connected to the activated word line; and in step S224, performs a series of steps to accumulate the result of the first logical AND on a bit line basis.
[0106] At this time, the first and second input characteristic values (I1, I2) refer to the input characteristic values to be operated on, and are another representation of a single bit of input data. Meanwhile, the 'input characteristic value' multiplied by the weight in step S223, which operates the logical AND, refers to a single bit value for the 'input' in step S221, where pre-charging takes place, and step S222, where activation takes place. That is, the 'input characteristic value' is a single bit, and the 'input' refers to a multi-bit value formed by combining 'input characteristic values' at multiple bit positions. For example, when the 'input' is 0111, the 'input characteristic value' is processed in the order of 1, 1, 1, 0.
[0107] Meanwhile, by repeatedly applying steps S221 to S224 to any second LAG different from the first LAG multiple times, it is possible to output multiple digital CIM calculation results within a single operation cycle for performing the analog CIM calculation step.
[0108] In this case, different LAGs process different input values. Therefore, the input characteristic values input to the first LAG and the second LAG, respectively, are different values, and they can be processed in parallel.
[0109] In step S230, the result of the analog CIM operation of step S210 and the result of the digital CIM operation of step S220 are accumulated.
[0110] When the CIM operation is completed in this manner, in step S300, the summer (130) of the DNN inference processor (100) aggregates the operation results of each of the multiple CIM operators (120) and stores them in the output memory (140), and in step S400, the precision encoding unit (150) converts the operation results into a mixed precision input.
[0111] In the description of the method of the present invention with reference to FIG. 3 and FIG. 14 to 16, redundant descriptions regarding the content mentioned in the description of the mixed precision DNN inference processor (100) of the present invention with reference to FIG. 3 to 13 have been omitted.
[0112] As explained above, the in-memory computing processor and method provided by the present invention have the characteristic of being able to improve the efficiency of mixed-precision computation of a DNN by preventing throughput limitations caused by outliers and minimizing ADC overhead when applying mixed-precision computation.
[0113] In addition, the in-memory computing processor and method provided in the present invention have the characteristic of improving the efficiency of mixed-precision computing of a DNN by processing the accumulation of high-bit outliers that are sparse and irregularly distributed during mixed-precision computing using digital CIM computing instead of conventional analog CIM computing, and by processing analog CIM computing and digital CIM computing in parallel.
[0114] In addition, the in-memory computing processor and method provided in the present invention have the characteristic of being able to reduce power waste for operating the ADC and improve computation speed by processing the processing result of the digital CIM operation in a digital peripheral circuit rather than the ADC.
[0115] In addition, the in-memory computing processor and method provided in the present invention are characterized by adopting a hierarchical cell array structure comprising a plurality of local-cell array groups (LAGs) and by processing each of the LAGs in parallel, thereby improving digital CIM computation efficiency.
[0116] In addition, the in-memory computing processor and method provided in the present invention have the characteristic of improving CIM throughput per unit time by performing two multiplications and one addition in a single cycle, by pre-charging the value of a bit line as a logical AND of a first and second input characteristic values, activating a first word line applied to an arbitrary first LAG as the first input characteristic value, and activating a second word line applied to an arbitrary second LAG different from the first LAG as the second input characteristic value.
[0117] Furthermore, the present invention can be applied to all fields requiring DNN inference on mobile devices. In particular, it can be utilized in applications that require both low power consumption and high accuracy simultaneously.
[0118] In particular, as machine learning has advanced rapidly, many hardware applications utilizing machine learning have been developed. However, hardware utilizing machine learning faces resource constraints such as form factor, battery power, and storage space. Consequently, there is a need for low-power machine learning hardware, and examples of such hardware include AIoT, TinyML, edge computing, and mobile devices.
[0119] Therefore, the present invention can be utilized in applications requiring image classification inference with high accuracy and low power consumption on mobile devices, etc., and thus the added value that can be extracted by the present invention is significant and the marketability is considered high.
[0120] Although preferred embodiments of the present invention have been presented and described in the above description, the present invention is not necessarily limited thereto, and those skilled in the art will readily understand that various substitutions, modifications, and changes can be made within the scope of the technical concept of the present invention. Explanation of the symbols
[0121] 100: DNN Inference Processor 110: Input Memory 120: CIM operators 121: DAS buffer 122: Analog decoder 123: SAS buffer 124: Digital decoder 125: Accumulator 130: Summer 140: Output memory 150: Precision Encoding Unit 160: Top-Controller 200: Cell Array Operation Unit 210: Local Array Group 211: Local cell array 300: Cell 310: 6T cell 320: Transistor 330: Cell capacitor
Claims
Claim 1 A DNN inference processor comprising: an input memory that stores data input for mixed-precision computation, wherein low-bit inliers and high-bit outliers are stored separately; a plurality of CIM operators that perform Computing In Memory (CIM) computations, wherein the inliers are subjected to analog CIM computation and the outliers are subjected to digital CIM computation, and then summed to produce an output; an aggregator that aggregates the computation results of each of the plurality of CIM operators; an output memory that stores the computation results of the aggregator; and a precision encoding unit that converts the computation results stored in the output memory into mixed-precision inputs. Claim 2 A DNN inference processor according to claim 1, wherein the CIM operator comprises a plurality of memory cells storing weights and a cell array operation unit that performs at least one of bit-unit analog CIM operation for an inlier existing within a predetermined operation cycle and bit-unit digital CIM operation for an outlier existing within the operation cycle; and an accumulation operator that accumulates the results of the analog CIM operation and the digital CIM operation. Claim 3 A DNN inference processor according to claim 2, wherein the cell array operation unit processes the analog CIM operation and the digital CIM operation in parallel by sharing a predetermined number of weights among those referenced for the analog CIM operation during the digital CIM operation. Claim 4 A DNN inference processor according to claim 2, wherein the cell array operation unit adopts a hierarchical cell array structure comprising a plurality of local-cell array groups (LAGs) composed of a predetermined number of memory cells, and processes the operation of each LAG in parallel. Claim 5 A DNN inference processor according to claim 4, wherein the cell array operation unit, during the digital CIM operation, pre-charges the value of a bit line applied to an arbitrary first LAG as a logical AND of arbitrary first and second input characteristic values (I1, I2), then activates a first word line applied to the first LAG as the first input characteristic value (I1) to generate a first logical AND result and a first logical OR result, and accumulates the input and weighted product using the first logical AND result and the first logical OR result. Claim 6 A DNN inference processor according to claim 5, wherein the cell array operation unit, during the digital CIM operation, pre-charges the value of a bit line applied to an arbitrary second LAG different from the first LAG as a logical AND of arbitrary third and fourth input characteristic values (I3, I4), then activates a second word line applied to the second LAG with the third input characteristic value (I3) to generate a second logical AND result and a second logical OR result, and accumulates the input and weighted product using the second logical AND result and the second logical OR result to produce a digital CIM operation result of the second LAG that is different from the digital CIM operation result of the first LAG. Claim 7 A DNN inference processor according to claim 2, wherein the memory cell comprises: a 6T SRAM cell operating via a bit-line, a bit-line bar, and a word line; a plurality of transistors coupled to an operation word line (CWL) and an operation bit line (CBL); and a cell capacitor (C) connected between the operation bit line (CBL) and the plurality of transistors. Claim 8 A mixed-precision DNN inference method using a DNN inference processor comprising a plurality of CIM operators, wherein the DNN inference processor stores data input for mixed-precision calculation, and separates and stores low-bit inliers and high-bit outliers; a CIM calculation step wherein each of the plurality of CIM operators performs a CIM calculation using the stored data, wherein the inliers are calculated using an analog CIM method and the outliers are calculated using a digital CIM method, and then summed and output; an aggregation step for aggregating the calculation results of each of the plurality of CIM operators; and a precision encoding step for converting the calculation result of the aggregation step into a mixed-precision input. Claim 9 A DNN inference method according to claim 8, wherein the above CIM operation step comprises: an analog CIM operation step that performs bit-unit accumulation of inliers located within a predetermined operation cycle and outputs the result after ADC conversion; a digital CIM operation step that performs bit-unit operation on outliers located within the operation cycle and outputs the result after accumulation by logical operation; and an accumulation operation step that accumulates the result of the analog CIM operation and the result of the digital CIM operation. Claim 10 A DNN inference method according to claim 9, wherein the digital CIM operation step is stored in a memory cell constituting the CIM operator and shares a predetermined number of weights referenced in the analog CIM operation step, thereby being processed in parallel with the analog CIM operation step. Claim 11 A DNN inference method according to claim 9, wherein the above CIM operation step is processed in parallel in units of a plurality of local cell array groups (LAG) composed of a predetermined number of memory cells. Claim 12 A DNN inference method according to claim 11, wherein the digital CIM operation step comprises: a first pre-charge step for pre-charging the value of a bit line applied to an arbitrary first LAG as a logical AND of arbitrary first and second input characteristic values (I1, I2); a first activation step for activating a first word line applied to the first LAG with either of the first or second input characteristic values (I1 or I2); a first logical AND operation step for generating a first logical AND, which is a product of the input and weights, in a memory cell connected to the activated word line; and a first logical OR operation step for accumulating the result of the first logical AND on a bit line basis. Claim 13 In claim 12, the digital CIM operation step further comprises: a second pre-charge step for pre-charging the value of a bit line applied to an arbitrary second LAG different from the first LAG as a logical AND of arbitrary third and fourth input characteristic values (I3, I4); a second activation step for activating a second word line applied to the second LAG with either the third or fourth input characteristic value (I3 or I4); a second logical AND operation step for generating a second logical AND, which is a product of the input and weights, in a memory cell connected to the activated word line; and a second logical OR operation step for accumulating the second logical AND result in bit line units, thereby outputting a plurality of digital CIM operation results within a single operation cycle of performing the analog CIM operation step.
Citation Information
Patent Citations
Low power high performance deep-neural-network learning accelerator and acceleration method
KR1020220101418A
In-memory accelerator for layer-wise quantized neural networks and operation method thereof
KR1020220116837A
