Circuit, method, and apparatus for in-memory computation

Abstract

Certain states provide means for performing machine learning tasks (and specifically, in-memory computation architectures). One state provides a circuit for in-memory computation. In general, the circuit includes: a plurality of memory cells on each of a plurality of columns of memory, the plurality of memory cells being configured to store a plurality of bits representing weights of a neural network, wherein the plurality of memory cells on each of the plurality of columns are on different word lines of the memory; a plurality of adder circuits, each adder circuit coupled to a corresponding column of the plurality of columns; a first adder circuit coupled to the outputs of at least two of the plurality of adder circuits; and an accumulator coupled to the output of the first adder circuit.

Description

Technical Field

[0001] This patent application claims priority to U.S. Application No. 17 / 391,718, filed August 2, 2021, which has been assigned to the assignee of this application and is incorporated herein by reference in its entirety.

[0002] The variants of this case concern the execution of machine learning tasks, and more specifically, the variants of this case concern computational architecture within memory. Prior Technology

[0003] Machine learning is typically the process of generating trained models (e.g., artificial neural networks, trees, or other structures) that represent a generalized fit to a set of prior, known training data. Applying the trained model to new data produces inferences, which can then be used to gain insights into the new data. In some cases, applying a model to new data is described as "performing inference" on the new data.

[0004] With the surge in the use of machine learning for various machine learning (or artificial intelligence) tasks, there has been a demand for more efficient processing of machine learning model data. In some cases, dedicated hardware, such as machine learning accelerators, can be used to enhance the processing system's ability to process machine learning model data. However, such hardware requires space and power, which are not always available on the processing device. For example, "edge processing" devices (such as mobile devices, always-on devices, Internet of Things (IoT) devices, etc.) typically have to balance processing power with power and packaging constraints. Furthermore, accelerators may need to move data across a shared data bus, which can result in significant power consumption and introduce latency into other processes on the shared data bus. Therefore, alternative approaches to processing systems for processing machine learning model data are being considered.

[0005] Memory devices are another type of processing system that can be used to process machine learning model data via a process known as in-memory computation (CIM). Conventional CIM processes use analog signals to perform computations, which can lead to inaccurate results and negatively impact neural network computations. Therefore, systems and methods are needed to perform in-memory computations with increased accuracy. Summary of the Invention

[0006] Some models provide devices and techniques for performing machine learning tasks (and specifically, in-memory computational architectures).

[0007] One example provides a circuit for computation within a memory. In general, the circuit includes: a plurality of memory cells in each of a plurality of columns of the memory, the plurality of memory cells being configured to store a plurality of bits representing weights of a neural network, wherein the plurality of memory cells in each of the plurality of columns are on different word lines of the memory; a plurality of adder circuits, each adder circuit coupled to a corresponding column of the plurality of columns; a first adder circuit coupled to the outputs of at least two of the plurality of adder circuits; and an accumulator coupled to the output of the first adder circuit.

[0008] One approach provides a method for computation within memory. In summary, the method includes the following steps: adding output signals on corresponding columns of a plurality of columns of memory via each of a plurality of adding circuits, wherein a plurality of memory cells in each of the plurality of columns store a plurality of bits representing weights of a neural network, wherein the plurality of memory cells in each of the plurality of columns are on different word lines of the memory; adding the output signals of at least two of the plurality of adding circuits via a first adder circuit; and accumulating the output signal of the first adder circuit via an accumulator.

[0009] One approach provides an apparatus for computation within memory. In general, the apparatus includes: a first component for adding output signals on corresponding columns of a plurality of columns of memory, wherein a plurality of memory cells are located on each of the plurality of columns, the plurality of memory cells storing a plurality of bits representing weights of a neural network, wherein the plurality of memory cells on each of the plurality of columns are on different word lines of the memory; a second component for adding the output signals of at least two of the first component for addition; and a component for accumulating the output signals of the second component for addition.

[0010] Other forms provide: a processing system configured to perform the methods described above and herein; a non-transitory computer-readable medium including instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the methods described above and herein; a computer program product embodied on a computer-readable storage medium, including code for performing the methods described above and herein; and a processing system including components for performing the methods described above and herein.

[0011] The following description and related figures illustrate in detail certain illustrative features of one or more states. Simple Explanation of the Diagram

[0012] To gain a full understanding of the aforementioned features of this invention, a more specific description of the invention, briefly summarized above, can be obtained by referring to various embodiments (some of which are illustrated in the accompanying drawings). However, it should be noted that the drawings only illustrate certain typical embodiments of this invention and are therefore not intended to limit the scope of this invention, as other equally valid embodiments are permissible in this description.

[0013] Figures 1A-1D illustrate examples of various types of neural networks that can be implemented using the various forms of the content of this case.

[0014] Figure 2 illustrates an example of a conventional spin operation that can be implemented using the various forms of the content in this case.

[0015] Figures 3A and 3B illustrate examples of depth-separable spin operations that can be implemented using the various forms of the content of this case.

[0016] Figure 4 illustrates an exemplary memory cell implemented as an octet (8T) static random access memory (SRAM) cell for in-memory computing (CIM) circuitry.

[0017] Figure 5A illustrates a circuit for CIM in certain states according to the contents of this case.

[0018] Figure 5B illustrates an exemplary implementation of an adder circuit.

[0019] Figure 5C illustrates an exemplary implementation of an accumulator.

[0020] Figure 6 illustrates a circuit for CIM implemented using a bit column adder tree, based on certain states of the case.

[0021] Figure 7 is a timing diagram illustrating the signals associated with the circuit of Figure 6 according to certain states of the present case.

[0022] Figures 8A, 8B, and 8C are block diagrams illustrating weighted, configurable bit-sized CIM circuits of certain states according to the content of this case.

[0023] Figure 9 is a flowchart illustrating exemplary operations for memory computation in certain states according to the content of this case.

[0024] Figure 10 illustrates an exemplary electronic device configured to perform signal processing operations in a neural network, according to certain aspects of the present case.

[0025] To facilitate understanding, where possible, the same element symbols have been used to designate common elements for the purposes of the accompanying drawings. It is anticipated that elements and features disclosed in one specification can be beneficially incorporated into other specifications without further description. Implementation

[0026] Various embodiments of this document provide apparatus, methods, processing systems, and computer-readable media for performing in-memory computation (CIM) to handle data-intensive processing, such as implementing machine learning models. Some embodiments provide techniques for performing digital CIM using adder circuits, each adder circuit adding (e.g., accumulating) the output signals on a corresponding column of a plurality of columns of memory after multiple computation cycles. As used herein, "adder circuit" generally refers to any circuit that adds (or accumulates) the output signals of memory cells in a column across sequential computation cycles. In some cases, the adder circuit may be an accumulator. An accumulator generally refers to a circuit system for accumulating output signals across multiple cycles. In other cases, the adder circuit may be an adder tree. "Adder circuit" or "adder tree" generally refers to a digital adder for adding the output signals of multiple memory cells (e.g., memory cells across word lines or columns). An exemplary implementation of an adder circuit is described herein with respect to Figure 5B, and an exemplary implementation of an accumulator is described herein with respect to Figure 5C. Adder circuits can be implemented as adder trees or accumulators with multiple adder circuits. In some cases, word lines of the CIM circuit system are enabled sequentially, and the accumulator performs accumulation simultaneously to provide an accumulation result after two or more word lines have been enabled sequentially.

[0027] Some styles offer a folded architecture that allows for configurability of the bit size of the weights used for computation. For example, one or more processing paths (also known as "wings") of the CIM architecture can be disabled to adjust the bit size of the weights being used. For instance, eight processing paths (e.g., including columns and associated processing circuitry) can be used to implement eight-bit weights, or four processing paths (where the other four processing paths are temporarily disabled) can be used to implement four-bit weights.

[0028] CIM-based machine learning (ML) / artificial intelligence (AI) can be used for a wide variety of tasks, including image and audio processing, and making wireless communication decisions (e.g., optimizing or at least increasing throughput and signal quality). Furthermore, CIM can be based on various types of memory architectures (such as Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM) (e.g., based on SRAM cells as shown in Figure 4), Magnetoresistive Random Access Memory (MRAM), and Resistive Random Access Memory (ReRAM or RRAM)) and can be attached to various types of processing units, including Central Processing Units (CPUs), Digital Signal Processors (DSPs), Graphics Processing Units (GPUs), Field Programmable Gate Arrays (FPGAs), AI accelerators, and so on. Typically, CIM can beneficially reduce the "memory wall" problem, where moving data into and out of memory consumes more power than the computation of that data. Therefore, significant power savings can be achieved by performing computations in memory. This is particularly useful for various types of electronic devices, such as low-power edge processing devices, mobile devices, etc.

[0029] For example, a mobile device may include a memory device configured to store data and perform computational operations within memory. The mobile device may be configured to perform ML / AI operations based on data generated by the mobile device, such as image data generated by the mobile device's camera sensor. Therefore, the mobile device's memory controller unit (MCU) can load weights from another onboard memory (e.g., flash memory or RAM) into the memory device's CIM array and allocate input feature buffers and output (e.g., output enable) buffers. The processing device can then begin processing the image data by loading layers, for example, into the input buffers and processing those layers using the weights loaded into the CIM array. This processing can be repeated for each layer of the image data, and the outputs (e.g., output enable) can be stored in the output buffers and subsequently used by the mobile device for ML / AI tasks, such as face recognition. A brief background on neural networks, deep neural networks, and deep learning.

[0030] Neural networks are organized into layers of interconnected nodes. Typically, nodes (or neurons) are where computations are performed. For example, a node can combine input data with a set of weights (or coefficients) that amplify or suppress the input data. Thus, the amplification or suppression of an input signal can be viewed as an allocation of the relative importance of various inputs to the task the network is trying to learn. Typically, the products of the input weights are summed (or accumulated), and this sum is then passed via the node's enable function to determine whether the signal should proceed further in the network and to what extent.

[0031] In its most basic implementation, a neural network can have an input layer, hidden layers, and an output layer. "Deep" neural networks typically have more than one hidden layer.

[0032] Deep learning is a method for training deep neural networks. Typically, deep learning maps the network's inputs to its outputs, and is therefore sometimes called a "universal approximator" because it can learn an unknown function f(x) = y that approximates any input x and any output y. In other words, deep learning finds the correct f to transform x into y.

[0033] More specifically, deep learning trains nodes in each layer based on different sets of features derived from the output of the previous layer. Thus, with each successive layer of a deep neural network, the features become increasingly complex. Deep learning is powerful because it can progressively extract higher-level features from the input data and perform complex tasks (such as object recognition) by learning to represent the input at progressively higher levels of abstraction in each layer, thereby building useful feature representations of the input data.

[0034] For example, when using visual data for presentation, the first layer of a deep neural network can learn to recognize relatively simple features in the input data, such as edges. In another example, when using auditory data for presentation, the first layer of a deep neural network can learn to recognize the spectral power of specific frequencies in the input data. Subsequently, the second layer of the deep neural network can learn to recognize combinations of features based on the output of the first layer, such as simple shapes for visual data or sound combinations for auditory data. Higher layers can then learn to recognize complex shapes in visual data or words in auditory data. Even higher layers can learn to recognize common visual objects or spoken phrases. Therefore, deep learning architectures can perform particularly well when applied to problems with a natural hierarchical structure. Layer connectivity in neural networks

[0035] Neural networks (such as deep neural networks (DNNs)) can be designed with various connectivity patterns between layers.

[0036] Figure 1A illustrates an example of a fully connected neural network 102. In the fully connected neural network 102, each node in the first layer transmits its output to each node in the second layer, such that each node in the second layer receives input from each node in the first layer.

[0037] Figure 1B illustrates an example of a locally connected neural network 104. In the locally connected neural network 104, nodes in the first layer can connect to a finite number of nodes in the second layer. More generally, the locally connected layers of the locally connected neural network 104 can be configured such that each node in a layer will have the same or similar connectivity pattern, but with connectivity strengths (or weights) that can have different values ​​(e.g., values ​​associated with local regions 110, 112, 114, and 116 of the first-layer nodes). The connectivity patterns of locally connected networks can create spatially different receptive fields in higher layers because higher-layer nodes in a given region can receive inputs of attributes that have been trained to a restricted portion of the total input to the network.

[0038] One type of locally connected neural network is the spiral neural network (CNN). Figure 1C illustrates an example of a spiral neural network 106. The spiral neural network 106 can be configured such that the connection strength associated with the input for each node in the second layer is shared (e.g., for a local region 108 that overlaps with another local region of the first layer nodes). Spiral neural networks are well-suited for problems in which the spatial location of the input is meaningful.

[0039] One type of spiral neural network is the deep spiral network (DCN). A deep spiral network is a network with multiple spiral layers, which can also be configured with, for example, pooling and regularization layers.

[0040] Figure 1D illustrates an example of DCN 100, which is designed to recognize visual features in image 126 generated by image capturing device 130. For example, if image capturing device 130 is a camera mounted in or on a vehicle (or otherwise moves with the vehicle), DCN 100 can be trained using various supervised learning techniques to recognize traffic signs and even numerical values ​​on traffic signs. DCN 100 can also be trained for other tasks, such as recognizing lane markings or traffic lights. These are just some of the exemplary tasks, and many other tasks are possible.

[0041] In the example of Figure 1D, DCN 100 includes a feature extraction part and a classification part. After receiving image 126, cyclotron layer 132 applies a cyclotron core (e.g., as illustrated and described in Figure 2) to image 126 to produce a first feature map set (or intermediate enable) 118. Typically, a "core" or "filter" comprises a multidimensional array of weights designed to emphasize different modes of the input data channels. In various instances, "core" and "filter" can be used interchangeably to refer to the set of weights applied in the cyclotron neural network.

[0042] Subsequently, the first feature map set 118 can be resampled by a pooling layer (e.g., a max pooling layer, not shown) to produce a second feature map set 120. The pooling layer can reduce the size of the first feature map set 118 while retaining most of the information, thereby improving model performance. For example, the second feature map set 120 can be sampled from a 28x28 matrix to a 14x14 matrix by the pooling layer.

[0043] This process can be repeated through multiple layers. In other words, the second feature map set 120 can be further spun through one or more subsequent spun layers (not shown) to produce one or more subsequent feature map sets (not shown).

[0044] In the example of Figure 1D, the second feature map set 120 is provided to the fully connected layer 124, which in turn produces an output feature vector 128. Each feature in the output feature vector 128 may include a numerical value corresponding to a possible feature of the image 126 (e.g., "sign", "60", and "100"). In some cases, a softmax function (not shown) can convert the numerical values ​​in the output feature vector 128 into probabilities. In this case, the output 122 of the DCN 100 is the probability that the image 126 includes one or more features.

[0045] The softmax function (not shown) converts the elements of the output feature vector 128 into probabilities so that the output 122 of DCN 100 is one or more probabilities that image 126 includes one or more features (such as a sign having the value "60" thereon, as in image 126). Therefore, in this example, the probabilities for "sign" and "60" in output 122 should be higher than the probabilities for other elements of output 122 (such as "30", "40", "50", "70", "80", "90", and "100").

[0046] Before training DCN 100, the output 122 generated by DCN 100 may be incorrect. Therefore, the error between output 122 and the prior known target output can be calculated. For example, here the target output is an indication that image 126 includes a "sign" and the value "60". Using the known target output, the weights of DCN 100 can then be adjusted via training so that subsequent outputs 122 of DCN 100 reach the target output (with high probability).

[0047] To adjust the weights of the DCN 100, the learning algorithm can compute gradient vectors for the weights. The gradient vectors indicate how much the error will increase or decrease if the weights are adjusted in a particular way. The weights can then be adjusted to reduce the error. This method of adjusting weights can be called "backpropagation" because it involves "passing back" through the layers of the DCN 100.

[0048] In practice, the error gradient of the weights can be calculated using a small number of instances, making the calculated gradient approximate the true error gradient. This approximation method is called stochastic gradient descent. Stochastic gradient descent can be repeated until the achievable error rate of the entire system has stopped decreasing or until the error rate has reached the target level.

[0049] After training, new images can be fed into the DCN 100, and the DCN 100 can generate inferences (such as classification) or probabilities of various features in the new images. Rotation technology for rotational neural networks

[0050] Coil processing is commonly used to extract useful features from a set of input data. For example, in a coil neural network such as the one described above, coil processing enables the extraction of different features using cores and / or filters whose weights are automatically learned during training. The extracted features are then combined for inference.

[0051] Enable functions can be applied before and / or after each layer of a spiral neural network. An enable function is typically a mathematical function that determines the output of a node in the neural network. Therefore, the enable function determines whether a node should pass information based on whether its input is relevant to the model's prediction. In one instance, in y = conv(x) (i.e., y is a spiral of x), both x and y can generally be considered "enabled". However, for a specific spiral operation, x can also be called "pre-enabled" or "input enabled" because x exists before that specific spiral, and y can be called output enabled or feature map.

[0052] Figure 2 illustrates an example of conventional spiral, where a 12-particle x 12-particle x 3-channel input image is spiraled using a 5x5x3 spiral core 204 and a stride (or stride size) of 1. The resulting feature map 206 is an 8-particle x 8-particle x 1-channel image. As can be seen in this example, conventional spiral can change the dimensionality of the input data (here, from 12x12 to 8x8 particles) (compared to the output data), including the channel dimension (here, from 3 channels to 1 channel).

[0053] One way to reduce computational overhead (e.g., measured in floating-point operations per second (FLOPs)) and the number of parameters associated with neural networks that include spiral layers is to decompose the spiral layers. For example, spatially separable spirals, such as those illustrated in Figure 2, can be decomposed into two components: (1) depthwise spirals, where each spatial channel is spiraled independently via depthwise spirals (e.g., spatial fusion); and (2) pointwise spirals, where all spatial channels are linearly combined (e.g., channel fusion). Examples of depthwise separable spirals are illustrated in Figures 3A and 3B. Typically, during spatial fusion, the network learns features from the spatial plane, and during channel fusion, the network learns relationships between these features across channels.

[0054] In one example, depth-separable spin can be implemented using a 5x5 core for spatial fusion and a 1x1 core for channel fusion. Specifically, channel fusion can use a 1x1xd core, which iteratively passes through each point of the input image at a depth of d, where the depth d of the core typically matches the number of channels in the input image. Channel fusion via point-by-point spin is useful for dimensionality reduction for efficient computation. Applying 1x1xd cores and adding an enable layer after the cores can provide increased network depth, which can improve network performance.

[0055] Specifically, in Figure 3A, a 12-particle x 12-particle x 3-channel input image 302 is cyclotronically processed using filters comprising three separate cores 304A-C, each having a 5x5x1 dimension, to produce an 8-particle x 8-particle x 3-channel feature map 306, where each channel is generated by a single core among the cores 304A-C.

[0056] Subsequently, feature map 306 is further rotated using a pointwise rotation operation, where a core 308 with dimensions 1x1x3 generates a feature map 310 with 8 primitives x 8 primitives x 1 channel. As illustrated in this example, feature map 310 has a reduced dimension (1 channel compared to 3 channels), which allows for more efficient computation.

[0057] Although the results of depth-separable spinners in Figures 3A and 3B are essentially similar to those of conventional spinners in Figure 2, the computational cost is significantly reduced, and thus depth-separable spinners provide a significant efficiency gain where network design allows.

[0058] Although not illustrated in Figure 3B, multiple (e.g., m) pointwise cyclotron cores 308 (e.g., individual components of a filter) can be used to increase the channel dimension of the cyclotron output. Thus, for example, m=256 1x1x3 cores 308 can be generated, each outputting an 8-primitive x 8-primitive x 1-channel feature map (e.g., feature map 310), and these feature maps can be stacked to obtain a resulting 8-primitive x 8-primitive x 256-channel feature map. This increase in channel dimension provides more parameters for training, which can improve the ability of the cyclotron neural network (e.g., in input image 302) to recognize features. Exemplary in-memory computing (CIM) architecture

[0059] Figure 4 illustrates an exemplary memory cell 400 of a static random access memory (SRAM) that can be implemented in a CIM array. The memory cell 400 may be referred to as an 8-transistor (8T) SRAM cell because the memory cell 400 is implemented using eight transistors.

[0060] As shown, memory cell 400 may include a cross-coupled inverter pair 424 having outputs 414 and 416. As shown, the cross-coupled inverter pair output 414 is selectively coupled to a write bit line (WBL) 406 via a transmission gate crystal 402, and the cross-coupled inverter pair output 416 is selectively coupled to a complementary write bit line (WBLB) 420 via a transmission gate crystal 418. WBL 406 and WBLB 420 are configured to provide complementary digital signals to be written into (e.g., stored therein) the cross-coupled inverter pair 424. WBL and WBLB can be used to store bits for neural network weights in memory cell 400. As shown, the gates of transmission gate crystals 402, 418 can be coupled to a write word line (WWL) 404. For example, the digital signal to be written can be provided to the WBL (and the complement of the digital signal can be provided to the WBLB). Subsequently, the transmission gate transistors 402 and 418 (implemented here as n-type field-effect transistors (NFETs)) are turned on by providing the logic high-level signal to WWL 404, thereby storing the digital signal in the cross-coupled inverter pair 424.

[0061] As shown in the figure, the cross-coupled inverter output 414 can be coupled to the gate of transistor 410. The source of transistor 410 can be coupled to a reference potential node (VSS or electrical ground), and the drain of transistor 410 can be coupled to the source of transistor 412. As shown in the figure, the drain of transistor 412 can be coupled to read bit line (RBL) 422. The gate of transistor 412 can be controlled via read word line (RWL) 408. RWL 408 can be controlled via an enable input signal.

[0062] During the read cycle, RBL 422 can be precharged to a logic high level. If both the enable input and the weight bit stored at the cross-coupled inverter pair output 414 are at a logic high level, both transistors 410 and 412 are turned on, electrically coupling RBL 422 to VSS at the source of transistor 410 and discharging RBL 422 to a logic low level. If either the enable input or the weight stored at the cross-coupled inverter pair output 414 is at a logic low level, at least one of transistors 410 and 412 is turned off, keeping RBL 422 at a logic high level. Therefore, the output of memory cell 400 at RBL 422 is at a logic low level only when both the weight bit and the enable input are at a logic high level, otherwise it is at a logic high level, thus effectively implementing NAND gate operation.

[0063] Figure 5A illustrates a circuit 500 for CIM according to certain configurations of this invention. Circuit 500 includes a CIM array 501 having word lines 504 0 to 504 31 (also referred to as rows) and columns 506 0 to 506 7. Word lines 504 0 to 504 31 are collectively referred to as word lines (WL) 504, and columns 506 0 to 506 7 are collectively referred to as columns 506. As shown, CIM array 501 may include an enable circuit system 590 configured to provide an enable signal to word lines 504. Although, for ease of understanding, CIM array 501 is implemented using 32 word lines and 8 columns, a CIM array can be implemented using any number of word lines or columns. As shown, memory cells 502 0-0 to 502 31-7 (collectively referred to as memory cells 502) are implemented at the intersection of WL 504 and column 506.

[0064] Each of the memory units 502 can be implemented using the memory unit architecture described with respect to Figure 4. As shown, enable inputs a(0,0) to a(31,0) can be provided to the corresponding word lines 504, and memory units 502 can store neural network weights w(0,0) to w(31,7). For example, memory units 502 0-0 to 502 0-7 can store weight bits w(0,0) to w(0,7), memory units 502 1-0 to 502 1-7 can store weight bits w(1,0) to w(1,7), and so on. Each word line can store multiple bits of weight. For example, weight bits w(0,0) to w(0,7) represent eight bits of the neural network weights.

[0065] As shown, circuit 500 may include adder trees 5100 to 5107 (collectively referred to as adder trees 510), each of which is implemented for a corresponding column in column 506. Each of adder trees 510 adds the output signals from memory cells 502 in the corresponding column of column 506. Each adder tree is implemented using an adder circuit tree (such as adder circuit 511). As shown, the output of adder tree 510 is coupled to weighted shift adder tree circuit 512. Weighted shift adder tree circuit 512 includes a plurality of weighted shift adders (e.g., weighted shift adder 514), each weighted shift adder including bit shift and add circuitry to facilitate the execution of bit shift and add operations. In other words, the memory cell on column 506 0 can store the most significant bit (MSB) for the corresponding weight, and the memory cell on column 506 7 can store the least significant bit (LSB) for the corresponding weight. Therefore, when an addition is performed across column 506, a bit shift operation is performed to shift those bits to take into account the importance of the bits on the associated column.

[0066] The output of the weighted shift adder tree circuit 512 is provided to the enabled shift accumulator circuit 516. The enabled shift accumulator circuit 516 includes a bit shifting circuit 518 and an accumulator 520. The enabled shift accumulator circuit 516 may also include flip-flops (FF) 522 and FF 591.

[0067] During the operation of circuit 500, enable circuit system 590 provides memory cell 502 with a first enable input set 599 a(0,0) to a(31,0) for computation during a first enable cycle. The first enable input set a(0,0) to a(31,0) represents the most significant bit of the enable parameter. The outputs of the computations on each column are summed using the corresponding one in adder tree 510. The outputs of adder tree 510 are summed using weighted shift adder tree circuit 512, and the result is provided to enable shift accumulator. During subsequent enable cycles, the same operation is performed for other enable input sets (such as enable inputs a(0,1) to a(31,1) representing the second most significant bit of the enable parameter), and so on, until the enable input representing the least significant bit of the enable parameter is processed. Bit shift circuit 518 performs bit shift operations based on the enable cycle. For example, for an 8-bit enable parameter processed using eight enable cycles, the bit shift circuit can perform an 8-bit shift in the first enable cycle, a 7-bit shift in the second enable cycle, and so on. After the enable cycle, the output of the bit shift circuit 518 is accumulated using accumulator 520 and stored in FF 522, 591, which can implement transfer registers.

[0068] Due to the symmetrical structure of the processing circuit system (such as the weighted shift adder tree circuit 512), the architecture of circuit 500 is referred to as a "folded" architecture. The folded architecture allows for configurability of the number of bits associated with the weights used during computation. For example, instead of computation using eight-bit weights, computation using four-bit weights can be implemented via four columns in deactivated column 506, as described in more detail herein.

[0069] The configuration described in Figure 5A provides bitwise storage and bitwise multiplication. Adder tree 510 performs group-counting addition for column 506. That is, each of the adder trees 510 adds the output signals of the memory cells used for one column. Weighted shift adder tree circuit 512 (e.g., for eight columns, having three stages as shown) combines the weighted sums generated for the eight columns (e.g., providing an accumulation result for the position of a given enabled bit during an enable cycle). Enable shift accumulator circuit 516 combines the results from multiple (e.g., eight) enable cycles and outputs the final accumulation result. For example, bit shift circuit 518 shifts the bit at the output of weighted shift adder tree circuit 512 based on the associated enable cycle. Serial accumulator 520 accumulates the shifted adder output generated by bit shift circuit 518. The transfer register implemented using FF 522 and 591 copies the output of serial accumulator 520 after the calculation for the last enabled cycle has been completed.

[0070] Parallel addition across columns improves the processing performance associated with circuit 500 (in trillions of operations per second (TOPS), provides a more compact full adder cell, reduces parasitic losses (because the adder is implemented next to the bit-multiplication memory cell), reduces switching activity (because fewer rows of memory have high enable amplitudes compared to conventional implementations), and provides a simple tiling for implementing simple macro generation (because the cells are placed side-by-side in an adjacency configuration to implement the adder tree). The configuration described in Figure 5A can be implemented using a single clock frequency.

[0071] Circuit 500 provides linear energy scaling across computation using enable or weight parameters of varying bit sizes. In other words, bit-size configurability is provided using adder tree 510 and weighted shift adder tree circuit 512, allowing n-bit enablement and m-bit weight accumulation, where n and m are positive integers, as described herein. The energy consumption associated with circuit 500 scales linearly based on the bit size configured for the enable parameter and weights.

[0072] Figure 5B illustrates an exemplary implementation of an adder circuit 585 configured to perform an addition operation. Adder circuit 585 may correspond to any adder circuit described herein, such as adder circuit 511. As shown, the adder circuit includes an XOR gate 573 that receives inputs 570 and 571 (labeled A and B). The output of XOR gate 573 is provided to an input of XOR gate 574, another input of which receives a carry input (Cin) signal 572. The output of XOR gate 574 provides the output of the adder circuit (labeled SUM). As shown, the adder circuit may also include an AND gate 575 that receives the Cin signal 572 and the output of XOR gate 573. AND gate 576 receives inputs 570 and 571. The outputs of AND gates 575 and 576 are provided to an input of OR gate 578, which generates a carry output signal for the addition operation. Although Figure 5B illustrates an exemplary implementation of the adder circuit for the purpose of facilitating understanding, the state described herein can be implemented using any suitable adder circuit architecture.

[0073] Figure 5C is an exemplary implementation of accumulator 587. Accumulator 587 can correspond to any accumulator described herein, such as accumulator 520. As shown, accumulator 520 includes adder circuitry 580 that receives an input signal. The output of adder circuitry 580 is provided to a register configured to store the output of the adder circuitry in each cycle of a clock signal provided to register 581. The output 582 of register 581 is used as the output of accumulator 587 and is fed back to the input of adder circuitry 580. Although Figure 5C illustrates one exemplary implementation of the accumulator for the purpose of understanding, the state described herein can be implemented using any suitable accumulator architecture.

[0074] Figure 6 illustrates a circuit 600 for CIM implemented using a bit column adder tree circuit 650 and a column accumulator circuit 652, according to certain states of the present invention. The bit column adder tree circuit 650 has inputs coupled to the outputs of multiple sense amplifiers 6020, 6021 through 6027, which are connected to multiple columns 506, each column having multiple bit lines (e.g., RBLs). For example, each column of 506 may have four bit lines, each bit line coupled to the input of one of the four sense amplifiers (e.g., sense amplifier 6020). The word lines of the CIM array 501 may include multiple word line groups (e.g., eight groups), each group having four word lines. Each of the four word lines in each group is coupled to a corresponding one of the four bit lines in each column. During a given computation cycle, each group of four word lines is activated via a corresponding enable signal, while the enable signals of the remaining word line groups are set to a logic low level. The word line groups are processed over a total of eight clock cycles (e.g., eight word line groups, totaling 32 word lines in this example). Sensing amplifiers 6020, 6021 through 6027 are collectively referred to as sensing amplifier 602. Multiple sensing amplifiers (e.g., four) are included for each column in column 506, allowing simultaneous sensing of multiple columns. For example, sensing amplifier 6020 simultaneously senses the outputs of corresponding memory cells 5020-0 through 5023-0 on column 5060, sensing amplifier 6021 simultaneously senses the outputs of corresponding memory cells 5020-1 through 5023-1 on column 5061, and so on, until sensing amplifier 6027 simultaneously senses the outputs of corresponding memory cells 5020-7 through 5023-7 on column 5067. The output of the sense amplifier for each column is coupled to the adder tree (e.g., adder trees 604 0, 604 1 to 604 7, collectively referred to as adder tree 604). Each adder circuit for implementing each of the adder trees 604 can be implemented as described with respect to FIG. 5B.

[0075] For simplicity, each of the sense amplifiers 602 is shown as having an input coupled to the output of a single memory cell. However, the input of each of the sense amplifiers 602 can be coupled to the output of multiple memory cells, which can be enabled sequentially. In other words, with four sense amplifiers for each column, four word lines can be enabled at a time on each column. As an example, as shown, the input of sense amplifier 602 0 can be coupled to the output of the corresponding memory cell for the first set of word lines (e.g., word lines 504 0 to 504 3), but also to the output of the corresponding memory cell for the second set of word lines (e.g., word lines 504 4 to 504 7), and to the output of the corresponding memory cell for the third set of word lines (e.g., word lines 504 8 to 504 11), and so on up to the last set of word lines (e.g., word lines 504 28 to 504 31). Therefore, for each column of 32 word lines and four sense amplifiers, eight computation cycles can be used to complete the computation for the set of enabled inputs (e.g., enabled inputs a(0,0) to a(31,0)).

[0076] As described, the output of adder tree 604 is coupled to column accumulator circuit 652. For example, the output of each of adder trees 604 is coupled to one of the accumulators 6060, 6061 through 6067 (collectively referred to as accumulators 606) of column accumulator circuit 652. Each of accumulators 606 can be implemented as described with respect to FIG. 5C. Each of accumulators 606 performs the accumulation of the output signal of the corresponding one of adder trees 604 across multiple computation cycles. For example, during each computation cycle, computation is performed for four word lines, and the output signals of the computation for the four word lines are added using adder tree 604 of bit column adder tree circuit 650. After multiple computation cycles (e.g., eight cycles for 32 word lines when using four sense amplifiers), each of accumulators 606 performs the accumulation of the output signal of the corresponding one of adder trees 604.

[0077] Once multiple computation cycles have been completed, the output of accumulator 606 is provided to weighted shift adder tree circuit 512 for column-wise addition, and the output of weighted shift adder tree circuit 512 is provided to enable shift accumulator circuit 516 for accumulation across enable cycles, as described with respect to FIG5A. In other words, bitwise accumulation occurs at each of the accumulators 606 across multiple computation cycles (e.g., eight computation cycles, each for four word lines, until computation for 32 word lines is completed). Weighted shift adder tree circuit 512 combines the weighted sums of the eight columns (e.g., providing the accumulation result for the position of a given enable bit during each enable cycle), and enable shift accumulator circuit 516 combines the results from multiple (e.g., eight) enable cycles to output the final accumulation result. In some configurations, the CIM array 501, bit column adder tree circuit 650, and column accumulator circuit 652 operate at a higher frequency (e.g., eight times, or less than eight times, when implemented using eight computation cycles, depending on the critical path delay limitation, while still using eight computation cycles) compared to the weighted shift adder tree circuit 512 and the enable shift accumulator circuit 516. As shown, half-latch circuits 6080, 6081 through 6087 (collectively referred to as half-latch circuit 608) can be coupled to the corresponding output of accumulator 606. Each half-latch circuit holds the output of the corresponding one of the accumulators 606 and provides the output to the corresponding input of the weighted shift adder tree circuit 512 once multiple computation cycles have been completed. In other words, a half-latch circuit typically refers to a latch circuit that holds a digital input (e.g., the output of one of the accumulators 606) at the beginning of a clock cycle and provides the digital input to the output of the latch circuit at the end of the clock cycle. Half-latch circuit 608 facilitates the transition from higher-frequency operation of column accumulator circuit 652 (e.g., 8x as shown) to lower-frequency operation of weighted shift adder tree circuit 512 (e.g., 1x as shown).

[0078] Figure 7 is a timing diagram 700 illustrating certain states of signals associated with circuit 600 according to the present invention. Circuit 600 can operate on a Digital Memory In-Process (DCIM) clock. The DCIM clock can be used as the master clock for circuits 500 and 600 to operate on. After eight cycles of the DCIM clock, a final accumulated output can be provided for multiplying the weights with the 8-bit enabled input. As shown, a higher frequency clock signal (referred to as the local clock) can be generated from a lower frequency DCIM clock. For example, the local clock can have a frequency eight times higher than the frequency of the DCIM clock.

[0079] As shown in the figure, one bit of each of the eight cycles of the DCIM clock is provided in each of the enable inputs. For example, during the first enable cycle (e.g., the first cycle of the DCIM clock), bits a(0,0) to a(31,0) (e.g., the MSBs of different enable inputs) are provided to the memory cell, during the second enable cycle (e.g., the second cycle of the DCIM clock), bits a(0,1) to a(31,1) (e.g., the second MSB (MSB-1) of different enable inputs) are provided to the memory cell, and so on.

[0080] During each cycle of the local clock, the output of sense amplifier 602 (labeled "SA Out") and the output of adder tree 604 (labeled "Col Add Out") are provided for the computation cycle. During each cycle of the local clock, SA Out and Col Add Out provide outputs for memory cells of a subset of word lines 504 (e.g., four word lines in the example described with respect to FIG. 6). For example, during the first cycle of the local clock, Col Add Out is provided for adder tree 604 0 for computations performed by memory cells 502 0-0 to 502 3-0, during the second cycle of the local clock, Col Add Out is provided for computations performed by memory cells 502 4-0 to 502 7-0, and so on, until Col Add Out is provided for computations performed by memory cells 502 28-0 to 502 31-0 (here, after 8 cycles of the local clock).

[0081] As shown in the figure, after eight local clock cycles (e.g., after a single DCIM clock cycle), the output of column accumulator circuit 652 (labeled "Col Acc Latch") and the output of weight shift adder tree circuit 512 (labeled "Weight Shift Add Out") are provided. Shift accumulator circuit 516 is enabled to accumulate Weight Shift Add Out across eight DCIM clock cycles and provides an output (labeled "Acc Out") at the end of the eight DCIM clock cycles.

[0082] In some configurations, the number of bits associated with enabling inputs and / or weights can be configurable. The bit column adder tree circuit 650 allows for configurability of the number of bits used for weights down to a single bit. For example, to implement 4-bit weights, the circuitry associated with columns 506 4, 506 5, 506 6, and 506 7 can be disabled, as described in more detail herein.

[0083] Figures 8A, 8B, and 8C are block diagrams illustrating a weighted, configurable bit-size CIM circuit system according to certain states of the present invention. For example, as shown in Figure 8A, 8-bit weights can be stored in memory cell 502 and processed using bit column adder tree circuit 650, column accumulator circuit 652, weight shift adder tree circuit 512, and enable shift accumulator circuit 516, as described herein.

[0084] As shown, clock generator circuit 870 may include clock generator 871 configured to generate a DCIM clock. Clock generator 871 may be implemented using any suitable clock generation circuit, such as a phase-locked loop (PLL) or a ring oscillator. Weighted shift adder tree circuit 512 may receive and operate on the DCIM clock described with respect to FIG7. For some cases, clock generator circuit 870 may include a frequency multiplier 802 that can be used to generate a local clock, enabling circuit system 590, bit column adder tree circuit 650, and column accumulator circuit 652 to operate based on this local clock. Although frequency multiplier 802 is shown as part of clock generator circuit 870, in some implementations, frequency multiplier 802 may be separate from clock generator 871. A frequency multiplier generally refers to any circuit that receives a clock signal having a first frequency and generates a second clock signal with a different second frequency, where the second frequency is a multiple of the first frequency.

[0085] Some examples provide computational techniques using wing-series operations, as described with respect to Figures 8B and 8C. As used herein, in the case of CIM circuits, a "wing-series operation" generally refers to an operation on one wing (one processing path of the CIM circuit) and subsequently on another wing (another processing path of the CIM circuit). For example, four-bit weights can be stored in memory cells on a set of four columns (e.g., columns 5064, 5065, 5066, 5067) for performing a first 4-bit weight calculation, and four-bit weights can be stored in memory cells on another set of four columns (e.g., columns 5060, 5061, 5062, 5063) for performing a second 4-bit weight calculation. These two sets of four columns can be disjoint sets. Although in the provided example, the 4-bit weight calculation is performed on columns 5064, 5065, 5066, and 5067 before the 4-bit weight calculation is performed on columns 5060, 5061, 5062, and 5063, the calculation can be performed in reverse order. For example, the 4-bit weight calculation can be performed on columns 5060, 5061, 5062, and 5063 before the 4-bit weight calculation is performed on columns 5064, 5065, 5066, and 5067.

[0086] Figure 8B illustrates the first cycle during which the first 4-bit weight calculation is performed. During the first cycle, the circuitry of the bit column adder tree circuit 650 and column accumulator circuit 652, which processes the signals for columns 5064, 5065, 5066, and 5067, is deactivated. For example, clock gate circuit 804 can be used to deactivate the clock supplied to accumulators 6064, 6065, 6066, and 6067 during the first cycle to reduce power consumption. As used herein, clock gate circuitry generally refers to any circuit that receives a clock signal (e.g., an AND gate having a first input that receives the clock signal) and, in response to a control signal, provides the clock signal to the output of the circuit (e.g., a control signal provided to the second input of the AND gate at a logic high level). At the end of the first cycle, shift accumulator circuit 516 is enabled to provide the result for the first 4-bit weight calculation.

[0087] Figure 8C illustrates the second cycle during which the second 4-bit weight calculation is performed. During the second cycle, the circuitry of the bit column adder tree circuit 650 and the column accumulator circuit 652, which processes the signals for columns 506 0, 506 1, 506 2, and 506 3, is deactivated. For example, clock gating circuit 804 can be used to deactivate the clock supplied to accumulators 606 0, 606 1, 606 2, and 606 3 during the second cycle to reduce power consumption. Although clock gating is illustrated only for the clock signal of column accumulator circuit 652 for the purpose of understanding, clock gating can be used to deactivate the clock signals of other unused circuits, such as the circuitry of the bit column adder tree. Exemplary operation for computing within digital memory (CIM)

[0088] Figure 9 is a flowchart illustrating an exemplary operation 900 for in-memory computation in certain forms according to the content of this case. Operation 900 can be performed by circuitry for CIM, such as circuitry 500 described with respect to Figure 5A or circuitry 600 described with respect to Figure 6.

[0089] Operation 900 begins at block 905 by the following operation: the circuit adds the output signals on a corresponding column of a plurality of columns (e.g., column 506) of the memory via each of a plurality of adder circuits (e.g., adder tree 510 or accumulator 606). A plurality of memory cells are located on each of the plurality of columns, and the plurality of memory cells store multiple bits representing the weights of the neural network (e.g., w(0,0) to w(31,7) shown in Figure 5A). The plurality of memory cells on each of the plurality of columns are located on different word lines (e.g., word line 504) of the memory.

[0090] At block 910, the circuit adds the output signals of at least two of the multiple adder circuits via a first adder circuit (e.g., weighted shift adder tree circuit 512). At block 915, the circuit accumulates the output signal of the first adder circuit via an accumulator (e.g., accumulator 520 or enable shift accumulator circuit 516). In some states, the circuit selectively disables one or more portions of the first adder circuit and / or one or more adder circuits based on the number of bits associated with each of the weights.

[0091] In some embodiments, adding the output signals of a corresponding column in a plurality of columns may include accumulating the output signals of memory cells in a corresponding column in a plurality of columns after two or more word lines in a plurality of word lines are sequentially enabled (e.g., via accumulator 606). In some embodiments, the circuitry adds the output signals of memory cells in a corresponding column in a plurality of columns and on two or more word lines via a second adder circuit (e.g., each adder tree of adder tree 604) coupled between each of the plurality of adder circuits and the corresponding column in a plurality of columns. In some embodiments, the circuitry senses the output signals of memory cells in a corresponding column in a plurality of columns and on two or more word lines via a sense amplifier (e.g., sense amplifier 602) coupled between the second adder circuitry and the corresponding column in a plurality of columns. In this case, the addition via the second adder circuitry is based on the sensed output signals.

[0092] In some states, the circuit disables a first portion of the first adder circuit and / or at least one adder circuit during a first calculation cycle, and disables a second portion of the first adder circuit and at least another adder circuit during a second calculation cycle.

[0093] In some configurations, the circuit sequentially enables two or more word lines. In this case, adding the output signals on a corresponding column of a plurality of columns via each of the plurality of adder circuits includes: after two or more word lines are sequentially enabled, accumulating the output signals of memory cells on a corresponding column of a plurality of columns via each of the plurality of adder circuits (e.g., accumulator 606).

[0094] In some embodiments, adding the output signals of at least two of the multiple adder circuits includes performing bit shifting and addition operations on at least two of the multiple adder circuits. In some embodiments, the circuit generates a first clock signal, wherein the multiple adder circuits operate based on the first clock signal (e.g., the local clock shown in FIG. 7), and the circuit generates a second clock signal, wherein the first adder circuit operates based on the second clock signal (e.g., the DCIM clock shown in FIG. 7), the second clock signal having a different frequency than the first clock signal. In some embodiments, the circuit generates the second clock signal based on the first clock signal via a frequency multiplier (e.g., frequency multiplier 802).

[0095] In some configurations, the circuit sequentially enables a plurality of memory cells based on different enable inputs, and the accumulation of the output signal of the first adder circuit occurs after the plurality of memory cells are sequentially enabled. For example, sequentially enabling a plurality of memory cells may include receiving a first set of enable inputs (e.g., enable inputs a(0,0) to a(31,0)) during a first enable cycle and receiving a second set of enable inputs (e.g., enable inputs a(0,1) to a(31,1)) during a second enable cycle, wherein the accumulation of the output signal of the first adder circuit occurs after the first and second enable cycles.

[0096] In some formats, multiple columns comprise a first subset of multiple columns (e.g., columns 506 0 to 506 3) and a second subset of multiple columns (e.g., columns 506 4 to 506 7). The first subset may be enabled during a first calculation cycle (e.g., cycle 1 as shown in Figure 8B). The second subset may be enabled during a second calculation cycle (e.g., cycle 2 as shown in Figure 8C), which follows the first calculation cycle.

[0097] In some configurations, memory cells on each word line are configured to store one weight in the weights of the neural network, and the number of a first subset of multiple columns (e.g., four in the example shown in Figure 8B) is associated with the number of bits in one weight. In some configurations, the circuitry disables the clock signal associated with processing the signal from the second subset of multiple columns via a clock gate circuit (e.g., clock gate circuit 804). Exemplary processing system for in-memory computation

[0098] Figure 10 illustrates an exemplary electronic device 1000. The electronic device 1000 can be configured to perform the methods described herein, including operation 900 described with respect to Figure 9.

[0099] Electronic device 1000 includes a central processing unit (CPU) 1002, which in some cases may be a multi-core CPU. Instructions executed at CPU 1002 may be loaded, for example, from program memory associated with CPU 1002 or from memory 1024.

[0100] The electronic device 1000 also includes additional processing blocks tailored for specific functions, such as a graphics processing unit (GPU) 1004, a digital signal processor (DSP) 1006, a neural processing unit (NPU) 1008, a multimedia processing block 1010, and a wireless connectivity processing block 1012. In one implementation, the NPU 1008 is implemented in one or more of the CPU 1002, GPU 1004, and / or DSP 1006.

[0101] In some embodiments, the wireless connectivity processing block 1012 may include elements for, for example, third-generation (3G) connectivity, fourth-generation (4G) connectivity (e.g., 4G LTE), fifth-generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and wireless data transmission standards. The wireless connectivity processing block 1012 is further connected to one or more antennas 1014 to facilitate wireless communication.

[0102] Electronic device 1000 may also include one or more sensor processors 1016 associated with any type of sensor, one or more image signal processors (ISP) 1018 associated with any type of image sensor, and / or navigation processor 1020, which may include satellite-based positioning system elements (e.g., GPS or GLONASS) and inertial positioning system elements.

[0103] Electronic device 1000 may also include one or more input and / or output devices 1022, such as a screen, a touch-sensitive surface (including a touch-sensitive display), a physical button, a speaker, a microphone, etc. In some embodiments, one or more processors in the processor of electronic device 1000 may be based on the ARM instruction set.

[0104] Electronic device 1000 also includes memory 1024, which represents one or more static and / or dynamic memories, such as dynamic random access memory, static memory based on flash memory, etc. In this example, memory 1024 includes computer executable elements that can be executed by one or more processors of the aforementioned processors of electronic device 1000 or by CIM controller 1032 (also referred to as control circuitry system). For example, electronic device 1000 may include CIM circuitry 1026 (such as circuitry 500), as described herein. CIM circuitry 1026 may be controlled via CIM controller 1032. For example, in some embodiments, memory 1024 may include code 1024A for storage (e.g., storing weights in memory cells) and code 1024B for computation (e.g., performing neural network computation via application-enabled input). As shown in the figure, the CIM controller 1032 may include circuitry 1028A for storage (e.g., storing weights in memory cells) and circuitry 1028B for computation (e.g., performing neural network computation via application enable input). The illustrated elements and other elements not shown may be configured to perform various states of the methods described herein.

[0105] In some cases (such as when electronic device 1000 is a server device), one or more of the following modules can be omitted from the example illustrated in FIG10: multimedia processing block 1010, wireless connection processing block 1012, antenna 1014, sensor processor 1016, ISP 1018, or navigation processor 1020. Exemplary terms

[0106] Clause 1. A circuit for computation within a memory, comprising: a plurality of memory cells in each of a plurality of columns of the memory, the plurality of memory cells being configured to store a plurality of bits representing weights of a neural network, wherein the plurality of memory cells in each of the plurality of columns are on different word lines of the memory; a plurality of adder circuits, each adder circuit coupled to a corresponding column of the plurality of columns; a first adder circuit coupled to the outputs of at least two of the plurality of adder circuits; and an accumulator coupled to the output of the first adder circuit.

[0107] Clause 2, the circuit according to Clause 1, wherein one or more portions of the first adder circuit are configured to be selectively disabled.

[0108] Clause 3. A circuit according to any one of Clauses 1-2, wherein each of the plurality of adder circuits includes an adder tree coupled to the plurality of memory cells in the corresponding column of the plurality of columns.

[0109] Clause 4. A circuit according to any one of Clauses 1-3, wherein each of the plurality of adder circuits includes another accumulator.

[0110] Clause 5. A circuit according to any one of Clauses 1-4, wherein a first portion of the first adder circuit is configured to be selectively disabled during a first calculation cycle, and wherein a second portion of the first adder circuit is configured to be selectively disabled during a second calculation cycle.

[0111] Clause 6. The circuit according to any one of Clauses 1-5 also includes: a second adder circuit coupled between each of the plurality of adder circuits and the corresponding column of the plurality of columns.

[0112] Clause 7. The circuit according to Clause 6, wherein the second adder circuit includes an adder tree coupled to two or more word lines in the word lines.

[0113] Clause 8. The circuit according to Clause 7, wherein the adder tree is configured to add the output signals of the memory cells on the respective column of the plurality of columns and on the two or more word lines of the plurality of word lines.

[0114] Clause 9. The circuitry pursuant to Clause 6 also includes: a sense amplifier coupled between the second adder circuitry and a corresponding column of the plurality of columns.

[0115] Clause 10. A circuit according to any one of Clauses 1-9, wherein the first adder circuit includes an adder tree configured to add the output signals of at least two of the plurality of adder circuits.

[0116] Clause 11. The circuit according to Clause 10, wherein one or more adders of the adder tree include bit shifting and adding circuits.

[0117] Clause 12. The circuit according to any one of Clauses 1-11 also includes: a clock generator circuit having a first output configured to output a first clock signal and a second output configured to output a second clock signal, wherein: the plurality of adder circuits are coupled to the first output of the clock generator and configured to operate based on the first clock signal; and the first adder circuit is coupled to the second output of the clock generator and configured to operate based on the second clock signal, the second clock signal having a different frequency than the first clock signal.

[0118] Clause 13. The circuit according to Clause 12, wherein the clock generator circuit includes a frequency multiplier configured to generate the second clock signal based on the first clock signal.

[0119] Clause 14. The circuit according to any one of Clauses 1-13 also includes: a plurality of half-latch circuits, each half-latch circuit being coupled between the first adder circuit and one of the plurality of adder circuits.

[0120] Clause 15. A circuit according to any one of Clauses 1-14, wherein: the plurality of memory cells are configured to be sequentially enabled based on different enable inputs; and the accumulator is configured to accumulate the output signal of the first adder circuit after the plurality of memory cells are sequentially enabled.

[0121] Clause 16. A circuit according to any one of Clauses 1-15, wherein the accumulator is the only accumulator coupled to the output of the first adder circuit.

[0122] Clause 17. A circuit according to any one of Clauses 1-16, wherein: the plurality of columns includes a first subset of the plurality of columns and a second subset of the plurality of columns; and the first subset is enabled during a first calculation cycle.

[0123] Clause 18, the circuit pursuant to Clause 17, wherein the second subset is enabled during a second calculation cycle, which follows the first calculation cycle.

[0124] Clause 19. A circuit according to any one of Clauses 17-18, wherein: at least some of the memory cells on each of the word lines are configured to store one of the weights of the neural network; and the number of the first subset of the plurality of columns is associated with the number of bits of the one weight among the weights.

[0125] Clause 20. The circuit according to any one of Clauses 17-19 also includes: a clock gate circuit having outputs coupled to the plurality of adder circuits and configured to disable the clock signal associated with processing the second subset of signals from the plurality of columns.

[0126] Clause 21. A method for computation within a memory, comprising the steps of: adding output signals on corresponding columns of a plurality of columns of the memory via each of a plurality of adding circuits, wherein a plurality of memory cells in each of the plurality of columns store a plurality of bits representing weights of a neural network, wherein the plurality of memory cells in each of the plurality of columns are on different word lines of the memory; adding output signals of at least two of the plurality of adding circuits via a first adder circuit; and accumulating the output signal of the first adder circuit via an accumulator.

[0127] Clause 22, the method pursuant to Clause 21, also includes the step of selectively disabling one or more portions of the first adder circuit based on the number of bits associated with each of the equal weights.

[0128] Clause 23. The method according to any one of Clauses 21-22, wherein adding the output signals on a corresponding column of the plurality of columns comprises: accumulating the output signals of the memory cells on the corresponding column of the plurality of columns after two or more word lines of the plurality of word lines are sequentially enabled.

[0129] Clause 24. The method according to Clause 23 also includes the following steps: adding the output signals of the memory cells on the corresponding column of the plurality of columns and the two or more word lines of the plurality of columns via a second adder circuit coupled between each of the plurality of adder circuits and the corresponding column of the plurality of columns.

[0130] Clause 25. The method according to Clause 24 also includes the following steps: sensing the output signals of the memory cells in the corresponding column of the plurality of columns and on the two or more word lines of the plurality of columns via a sensing amplifier coupled between the second adder circuit and the corresponding column of the plurality of columns, wherein the addition via the second adder circuit is based on the sensed output signals.

[0131] Clause 26. The method according to any one of Clauses 21-25, wherein adding the output signals of at least two of the plurality of adder circuits comprises: performing bit shifting and addition operations on the at least two of the plurality of adder circuits.

[0132] Clause 27. The method according to any one of Clauses 21-26 also includes the steps of: generating a first clock signal, wherein the plurality of adder circuits operate based on the first clock signal; and generating a second clock signal, wherein the first adder circuit operates based on the second clock signal, the second clock signal having a different frequency than the first clock signal.

[0133] Clause 28. The method according to any one of Clauses 21-27 also includes the following steps: sequentially enabling the plurality of memory cells based on different enable inputs, wherein the accumulation of the output signals of the first adder circuit occurs after the plurality of memory cells are sequentially enabled.

[0134] Clause 29. The method according to Clause 28, wherein sequentially enabling the plurality of memory cells includes: receiving a first set of enable inputs during a first enable cycle; and receiving a second set of enable inputs during a second enable cycle, wherein the accumulation of the output signals of the first adder circuit occurs after the first enable cycle and the second enable cycle.

[0135] Clause 30. An apparatus for computation within a memory, comprising: a first component for adding output signals on corresponding columns of a plurality of columns of the memory, wherein a plurality of memory cells are on each of the plurality of columns, the plurality of memory cells storing a plurality of bits representing weights of a neural network, wherein the plurality of memory cells on each of the plurality of columns are on different word lines of the memory; a second component for adding the output signals of at least two of the first component for addition; and a component for accumulating the output signals of the second component for addition. Additional considerations

[0136] The preceding description is provided to enable those skilled in the art to implement the various forms described herein. The examples discussed herein do not limit the scope, applicability, or forms set forth in the claims. Various modifications to these forms will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other forms. For example, the function and arrangement of the elements discussed can be changed without departing from the scope of this application. Various procedures or elements can be omitted, substituted, or added as appropriate in each example. For example, the described method can be performed in a different order than that described, and steps can be added, omitted, or combined. Furthermore, features described with respect to some examples can be combined into others. For example, an apparatus or a method can be implemented using any number of forms set forth herein. Moreover, the scope of this application is intended to cover such an apparatus or method implemented using structures, functions, or structures and functions other than or different from the various forms disclosed herein. It should be understood that any form of the disclosed content can be embodied by one or more elements of the claims.

[0137] As used herein, the term "exemplary" means "serving as an example, instance, or illustration." Any state described as "exemplary" in this document is not necessarily to be construed as being better or more advantageous than other states.

[0138] As used in this article, the phrase "at least one of the items" refers to any combination of those items, including a single member. As an example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination of multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0139] As used herein, the term "decision" encompasses a wide variety of actions. For example, "decision" can include calculation, operation, processing, deduction, investigation, examination (e.g., examining a table, database, or other data structure), ascertainment, and so on. Furthermore, "decision" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), and so on. Additionally, "decision" can include parsing, selecting, choosing, creating, and so on.

[0140] The methods disclosed herein include one or more steps or actions for implementing the methods. These method steps and / or actions can be interchanged without departing from the scope of the claim. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claim. Furthermore, the various operations of the methods described above can be performed by any suitable component capable of performing the corresponding function. Such components can include various hardware and / or software elements and / or modules, including but not limited to: circuits, application-specific integrated circuits (ASICs), or processors. Typically, in the presence of operations illustrated in the figures, these operations can have corresponding paired components plus functional elements with similar numbering. For example, components for addition can include adder trees (such as adder tree 510 or weighted shift adder tree 512) or accumulators (such as accumulator 606). Components for accumulation can include accumulators (such as enabled shift accumulator 516). Components for sensing can include SAs (such as SA 602).

[0141] The following claims are not intended to be limited to the forms shown herein, but are given the full scope consistent with the language of the claims. Within the claims, unless expressly stated otherwise, references to singular elements do not mean "one and only one," but rather "one or more." Unless expressly stated otherwise, the term "some" refers to one or more. No claim element is to be interpreted pursuant to Article 18, Paragraph 8 of the Implementing Regulations of the Patent Law, unless the element is expressly described using the phrase "component for..." or, in the case of a method claim, using the phrase "step for...". All structural and functional equivalents of the elements throughout the various forms described herein are expressly incorporated herein by reference, and are intended to be covered by the scope of the claims, such structural and functional equivalents being known to a person of ordinary skill or to be known later. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is expressly stated in the claims.

[0142] 30: Elements 50: Elements 60: numerical value 70: Elements 80: Elements 90: Elements 100:DCN 102: Fully Connected Neural Networks 104: Local Connectivity Neural Networks 106: Rotational Neural Network 108: Local area 110: Local area 112: Local area 114: Local area 116: Local area 118: First Feature Map Set 120: Second feature map set 122: Output 124: Fully Connected Layer 126: Image 128: Output feature vector 130: Image capturing device 132: Cyclone 204: Spinning Core 206: Feature Map 302: Input Image 304A-C: Core 306: Feature Map 308: Core 310: Feature Map 400: Memory Unit 402: Transmission Thyristor 404:WWL 406:WBL 408:RWL 410: Transistor 412: Transistor 414: Output 416: Output 418: Transmission Thyristor 420:WBLB 422:RBL 424: Inverter Pair 500: Circuit 501: CIM Array 511: Adder Circuit 512: Weighted Shift Adder Tree Circuit 514: Weighted Shift Adder 516: Enable shift accumulator circuit 518: Bit shifting circuit 520: Accumulator 522: Trigger (FF) 570: Input 571: Input 572: Cin signal 573: XOR gate 574: XOR gate 575: AND gate 576: AND gate 578:OR gate 580: Adder Circuit 581: Temporary Register 582: Output 585: Adder Circuit 587: Accumulator 590: Enable circuit system 591:FF 599: First Enabled Input Set 600: Circuitry for CIM 650: Bit Column Adder Tree Circuit 652: Column accumulator circuit 700: Timing Diagram 802: Frequency multiplier 804: Clock gate control circuit 870: Clock Generator Circuit 871: Clock Generator 900: Operation 905: Square 910: Square 915: Square 1000: Electronic devices 1002: CPU 1004: GPU 1006:DSP 1008: Neural Processing Unit (NPU) 1010: Multimedia Processing Block 1012: Wireless Connection Processing Block 1014: Antenna 1016: Sensor Processor 1018: Image Signal Processor (ISP) 1020: Navigation Processor 1022: Input and / or output devices 1024: Memory 1024A: Code 1024B: Code 1026: CIM Circuit 1028A: Circuit 1028B: Circuit 1032: CIM Controller 502 0-0: Memory unit 502 0-1: Memory unit 502 0-7: Memory Unit 502 1-0: Memory unit 502 1-1: Memory Unit 502 1-7: Memory Unit 502 3-0: Memory Unit 502 3-1: Memory Unit 502 31-0: Memory unit 502 31-7: Memory Unit 502 3-7: Memory Unit 504 0: Word line 504 1: Word line 504 3: Word Line 504 31: Word Line 506 0: column 506 1: Column 506 7: column 510 0: Adder Tree 510 7: Adder Tree 602 0: Sensing Amplifier 602 1: Sensing Amplifier 602 7: Sensing Amplifier 604 0: Adder Tree 604 1: Adder Tree 604 7: Adder Tree 606 0: Accumulator 606 1: Accumulator 606 7: Accumulator 608 0: Half-latch circuit 608 1: Half-latch circuit 608 7: Half-latch circuit C in: Carry Input RBL: Read Bit Line RWL: Reading line SUM: Output of the adder circuit WBL: Write Bit Line WBLB: Write Bit Line WWL: Writing Line

[0143] Domestic storage information (please note in order of storage institution, date, and number) none Overseas storage information (please note in the order of storage country, institution, date, and number) none

Claims

1. A circuit for computation within a memory, comprising: A plurality of memory cells on each of a plurality of columns of a memory, the plurality of memory cells being configured to store a plurality of bits representing the weights of a neural network, wherein the plurality of memory cells on each of the plurality of columns are on different word lines of the memory; a plurality of adder circuits, each adder circuit being coupled to a corresponding column of the plurality of columns; a first adder circuit being coupled to the outputs of at least two of the plurality of adder circuits; and an accumulator being coupled to an output of the first adder circuit, wherein one or more portions of the first adder circuit are configured to be selectively disabled.

2. The circuit according to claim 1, wherein each of the plurality of adder circuits includes an adder tree coupled to the plurality of memory cells in the corresponding column of the plurality of columns.

3. The circuit according to claim 1, wherein each of the plurality of adder circuits includes another accumulator.

4. The circuit according to claim 1, wherein a first portion of the first adder circuit is configured to be selectively disabled during a first calculation cycle, and wherein a second portion of the first adder circuit is configured to be selectively disabled during a second calculation cycle.

5. The circuit according to claim 1 also includes: A plurality of second adder circuits, wherein each of the second adder circuits is coupled between a corresponding adder circuit in the plurality of adder circuits and a corresponding column in the plurality of columns.

6. The circuit according to claim 5, wherein each of the second adder circuits comprises an adder tree coupled to two or more word lines in the word lines.

7. The circuit according to claim 6, wherein the adder tree is configured to add the output signals of the memory cells on the corresponding column of the plurality of columns and on the two or more word lines of the plurality of word lines.

8. The circuit according to claim 5 also includes: A sensing amplifier coupled between each of the second adder circuits and the corresponding column of the plurality of columns.

9. The circuit according to claim 1, wherein the first adder circuit includes an adder tree configured to add the output signals of at least two of the plurality of adder circuits.

10. The circuit according to claim 9, wherein one or more adders of the adder tree include a bit shift and adder circuit.

11. The circuit according to claim 1 also includes: A clock generator circuit has a first output configured to output a first clock signal and a second output configured to output a second clock signal, wherein: a plurality of adder circuits are coupled to the first output of the clock generator and configured to operate based on the first clock signal; and the first adder circuit is coupled to the second output of the clock generator and configured to operate based on the second clock signal, the second clock signal having a frequency different from the first clock signal.

12. The circuit according to claim 11, wherein the clock generator circuit includes a frequency multiplier configured to generate the second clock signal based on the first clock signal.

13. The circuit according to claim 1 also includes: A plurality of half-latch circuits, each half-latch circuit being coupled between the first adder circuit and one of the plurality of adder circuits.

14. The circuit according to request item 1, wherein: The plurality of memory cells are configured to be enabled sequentially based on different enable inputs; and the accumulator is configured to accumulate the output signal of the first adder circuit after the plurality of memory cells are enabled sequentially.

15. The circuit according to claim 1, wherein the accumulator is the only accumulator coupled to the output of the first adder circuit.

16. The circuit according to request item 1, wherein: The plurality of columns includes a first subset of the plurality of columns and a second subset of the plurality of columns; and the first subset is enabled during a first computation cycle.

17. The circuit according to claim 16, wherein the second subset is enabled during a second computation cycle after the first computation cycle.

18. The circuit according to request item 16, wherein: At least some of the memory cells on each of the word lines are configured to store one of the weights of the neural network; and a number of the first subset of the plurality of columns is associated with a number of bits of the one weight among the weights.

19. The circuit according to claim 16 also includes: A clock gate circuit having outputs coupled to the plurality of adder circuits and configured to disable a clock signal associated with processing the second subset of signals from the plurality of columns.

20. A method for computation within a memory, comprising the steps of: adding output signals on a corresponding column of a plurality of columns of a memory via each of a plurality of adding circuits, wherein a plurality of memory cells in each of the plurality of columns store a plurality of bits representing weights of a neural network, wherein the plurality of memory cells in each of the plurality of columns are on different word lines of the memory; adding output signals of at least two of the plurality of adding circuits via a first adder circuit; accumulating the output signal of the first adder circuit via an accumulator; and selectively disabling one or more portions of the first adder circuit based on a number of bits associated with each of the equal weights.

21. The method of claim 20, wherein the step of adding the output signals on the corresponding column of the plurality of columns includes the following steps: after two or more word lines of the plurality of word lines are sequentially enabled, the output signals of the memory cells on the corresponding column of the plurality of columns are accumulated.

22. The method according to claim 21 also includes the following steps: adding the output signals of the memory cells on the corresponding column of the plurality of columns and the two or more word lines of the plurality of word lines via a plurality of second adder circuits, wherein each of the second adder circuits is coupled between a corresponding adder circuit of the plurality of adder circuits and the corresponding column of the plurality of columns.

23. The method according to claim 22 also includes the step of: sensing the output signals of the memory cells on the corresponding column of the plurality of columns and the two or more word lines of the plurality of columns via a sensing amplifier coupled between each of the second adder circuits and the corresponding column of the plurality of columns, wherein the addition via the second adder circuits is based on the sensed output signals.

24. The method of claim 20, wherein the step of adding the output signals of at least two of the plurality of adder circuits includes the following steps: performing a bit shift and addition operation on the at least two of the plurality of adder circuits.

25. The method according to claim 20 also includes the steps of: generating a first clock signal, wherein the plurality of adder circuits operate based on the first clock signal; and generating a second clock signal, wherein the first adder circuit operates based on the second clock signal, the second clock signal having a frequency different from the first clock signal.

26. The method according to claim 20 also includes the step of sequentially enabling the plurality of memory cells based on different enable inputs, wherein the accumulation of the output signals of the first adder circuit occurs after the plurality of memory cells are sequentially enabled.

27. The method of claim 26, wherein the step of sequentially enabling the plurality of memory cells includes the following steps: receiving a first enable input set during a first enable cycle; and receiving a second enable input set during a second enable cycle, wherein the accumulation of the output signals of the first adder circuit occurs after the first enable cycle and the second enable cycle.

28. A device for computation within memory, comprising: A first component for adding output signals on a corresponding column of a plurality of columns of a memory, wherein a plurality of memory cells are on each of the plurality of columns, the plurality of memory cells storing a plurality of bits representing weights of a neural network, wherein the plurality of memory cells on each of the plurality of columns are on different word lines of the memory; a second component for adding the output signals of at least two of the first component for adding; a component for accumulating the output signals of the second component for adding; and a component for selectively disabling one or more portions of the first adder circuitry based on a number of bits associated with the weights of each of the equal weights.

Images