Reconstructing MAC operation

Abstract

A method and apparatus for performing a reconstructed multiplication and accumulation operation is provided. A summing array includes a plurality of nonvolatile memory elements arranged in columns. Each nonvolatile memory element in the summing array is programmed to a high resistance state or a low resistance state based on a weight of a neural network. The summing array is configured to generate a summed signal for each column based at least in part on a plurality of input signals. A multiplication array is coupled to the summing array and includes a plurality of nonvolatile memory elements. Each nonvolatile memory element in the multiplication array is programmed to a different conductance level based on a weight of the neural network. The multiplication array is configured to generate an output signal based at least in part on the summed signal from the summing array.

Description

Background Art

[0001] The present disclosure relates generally to machine learning (ML), and more particularly, to multiply and accumulate (MAC) operations for ML applications and processors.

[0002] A crossbar switch architecture based on non-volatile memory (NVM) provides an alternative mechanism for performing MAC operations in machine learning algorithms, particularly neural networks. A mixed-signal approach using NVM bit cells relies on Ohm's law to implement multiplication operations by exploiting the resistive properties of emerging NVM technologies (e.g., phase change memory (PCM), resistive random access memory (RRAM), correlated electronic random access memory (CeRAM), etc.). Applying a voltage bias across an NVM bit cell generates a current that is proportional to the product of the conductance of the NVM element and the voltage bias across the cell.

[0003] The currents from multiple bit cells are added in parallel to implement the cumulative sum. Thus, the combination of Ohm's law and Kirchhoff's current law implements multiple MAC operations in parallel. However, these can be energy intensive when implemented in the digital domain using explicit multipliers and adders.

[0004] MAC operations are utilized in ML applications such as artificial neural networks (ANNs), including deep neural networks (DNNs), convolutional neural networks (CNNs), etc. MAC acceleration using NVM crossbars requires programming NVM elements with precise conductance levels representing multi-bit weight parameters. Due to inherent device limitations, the representable bit precision is limited to 4 or 5 bits, which provides 16 to 32 different conductance levels. This complicates the weight programming step because the entire crossbar array of NVM bits needs to be accurately programmed (1Mb to 10Mb capacities are typical). BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present disclosure is illustrated by way of example, and not limitation, in the accompanying figures in which like reference numerals indicate similar elements.

[0006] Figure 1A An ANN according to an embodiment of the present disclosure is depicted.

[0007] Figure 1B A high-level block diagram of an ANN according to an embodiment of the present disclosure is depicted.

[0008] Figure 2A A CNN according to an embodiment of the present disclosure is depicted.

[0009] Figure 2B Depicted are convolution operations within a convolutional layer of a CNN according to an embodiment of the present disclosure.

[0010] Figure 2C Another convolutional layer operation of a CNN according to an embodiment of the present disclosure is depicted.

[0011] Figure 2D Depicted are the transformed convolutional layer operations of a CNN according to an embodiment of the present disclosure.

[0012] Figure 3A FIG. 1 shows an implementation of a NVM crossbar switch according to an embodiment of the present disclosure. Figure 2D Part of the convolutional layer operation.

[0013] Figure 3B Depicted is the architecture of a convolutional layer operation implemented using an NVM crossbar according to an embodiment of the present disclosure.

[0014] Figure 4A A schematic diagram of the architecture of a reconstructed MAC operation according to an embodiment of the present disclosure is depicted.

[0015] Figure 4B A schematic diagram of the architecture of a reconstructed MAC operation according to an embodiment of the present disclosure is depicted.

[0016] Figure 5 A flow chart of a low-precision programming process for reconstructed MAC operations according to an embodiment of the present disclosure is depicted.

[0017] Figure 6 A flow chart of a high precision programming process for a multiplier / scaling module is depicted in accordance with an embodiment of the present disclosure.

[0018] Figure 7 A flowchart of a method for performing multiply-accumulate acceleration in a neural network according to an embodiment of the present disclosure is depicted. DETAILED DESCRIPTION

[0019] The specific embodiments of the present disclosure will now be described in detail with respect to the accompanying drawings. For simplicity and clarity of explanation, reference numerals may be repeated in the accompanying drawings to indicate corresponding or similar elements where deemed appropriate. In addition, many specific details are described in order to provide a thorough understanding of the examples described herein. However, it will be appreciated by those of ordinary skill in the art that the examples described herein may be practiced without these specific details. In other cases, well-known methods, procedures, and components are not described in detail in order to avoid blurring the examples described herein. Moreover, this specification should not be considered to limit the scope of the examples described herein.

[0020] According to the present disclosure, an improved technique is provided for reconfiguring MAC operations to reduce programming steps in such systems as described above.

[0021] In one embodiment of the present disclosure, a device includes a summing array and a multiplying array. The summing array includes a plurality of nonvolatile memory elements arranged in columns. Each nonvolatile memory element in the summing array is programmed to a high resistance state or a low resistance state based on a weight of a neural network. The summing array is configured to generate a summed signal for each column based at least in part on a plurality of input signals. The multiplying array is coupled to the summing array and includes a plurality of nonvolatile memory elements. Each nonvolatile memory element in the multiplying array is programmed to a different conductance level based on a weight of a neural network. The multiplying array is configured to generate an output signal based at least in part on the summed signal from the summing array.

[0022] ANNs (such as DNNs, CNNs, etc.) are a popular solution to a wide array of challenging classification, recognition, and regression problems. However, many ANN models require a large number of computations involving a large number of weights and activations, which pose significant challenges in terms of access, storage, and performance, especially for mobile and other power or storage constrained devices. ANN hardware accelerators accelerate these computations, such as the convolution operations performed by CNNs.

[0023] ANN uses a network of interconnected nodes trained by a learning process to model the relationship between input data or signals and output data or signals. The nodes are arranged into various layers, including, for example, an input layer, one or more hidden layers, and an output layer. The input layer receives input data, such as image data, and the output layer generates output data, such as the probability that the image data contains a known object. Each hidden layer provides at least a partial transformation of the input data to the output data. DNN has multiple hidden layers to model the complex nonlinear relationship between the input data and the output data.

[0024] In a fully connected feedforward ANN, each node is connected to all nodes in the previous layer and to all nodes in the subsequent layers. For example, each input layer node is connected to each hidden layer node, each hidden layer node is connected to each input layer node and each output layer node, and each output layer node is connected to each hidden layer node. Additional hidden layers are similarly interconnected. Each connection has a weight value, and each node has an activation function, such as a linear function, a step function, a sigmoid function, a tanh function, a rectified linear unit (ReLU) function, etc., which determines the output of the node based on the weighted sum of the input to the node. The input data is propagated from the input layer nodes to the hidden layer nodes through the corresponding connection weights, and then propagated to the output layer nodes through the corresponding connection weights.

[0025] More specifically, at each input node, the input data is provided to the activation function of the node, and the output of the activation function is then provided to each hidden layer node as the input data value. At each hidden layer node, the input data value received from each input layer node is multiplied by the corresponding connection weight, and the resulting product is summed or accumulated into the activation value of the activation function provided to the node. Then, the output of the activation function is provided to each output layer node as the input data value. At each output layer node, the output data value received from each hidden layer node is multiplied by the corresponding connection weight, and the resulting product is summed or accumulated into the activation value of the activation function provided to the node. Then, the output of the activation function is provided as the output data. Additional hidden layers can be similarly configured to process data.

[0026] Figure 1A An ANN 10 is depicted according to an embodiment of the present disclosure.

[0027] ANN 10 includes an input layer 20, one or more hidden layers 30, 40, 50, etc., and an output layer 60. Input layer 20 includes one or more input nodes 21, 22, 23, etc. Hidden layer 30 includes one or more hidden nodes 31, 32, 33, 34, 35, etc. Hidden layer 40 includes one or more hidden nodes 41, 42, 43, 44, 45, etc. Hidden layer 50 includes one or more hidden nodes 51, 52, 53, 54, 55, etc. Output layer 60 includes one or more output nodes 61, 62, etc. Typically, ANN 10 includes N hidden layers, input layer 20 includes "i" nodes, hidden layer 30 includes "j" nodes, hidden layer 40 includes "k" nodes, hidden layer 50 includes "m" nodes, and output layer 60 includes "o" nodes.

[0028] In one embodiment, N is equal to 3, i is equal to 3, j, k and m are equal to 5 and o is equal to 2 (illustrated in FIG. 1 ). Input node 21 is coupled to hidden nodes 31 to 35, input node 22 is coupled to hidden nodes 31 to 35, and input node 23 is coupled to hidden nodes 31 to 35. Hidden node 31 is coupled to hidden nodes 41 to 45, hidden node 32 is coupled to hidden nodes 41 to 45, hidden node 33 is coupled to hidden nodes 41 to 45, hidden node 34 is coupled to hidden nodes 41 to 45, and hidden node 35 is coupled to hidden nodes 41 to 45. Hidden node 41 is coupled to hidden nodes 51 to 55, hidden node 42 is coupled to hidden nodes 51 to 55, hidden node 43 is coupled to hidden nodes 51 to 55, hidden node 44 is coupled to hidden nodes 51 to 55, and hidden node 45 is coupled to hidden nodes 51 to 55. Hidden node 51 is coupled to output nodes 61 and 62 , hidden node 52 is coupled to output nodes 61 and 62 , hidden node 53 is coupled to output nodes 61 and 62 , hidden node 54 is coupled to output nodes 61 and 62 , and hidden node 55 is coupled to output nodes 61 and 62 .

[0029] Figure 1B A high-level block diagram 100 of an ANN using MAC operations is depicted according to an embodiment of the present disclosure.

[0030] The MAC operation may be represented by Equation 1, where element b and element c are multiplied together, added to value a, stored in an accumulator, and then stored in the accumulator as a new value a.

[0031] a←a+(b·c) (Equation 1)

[0032] A set of MACs can represent dot products and vector-matrix multiplications. Figure 1B 100, such as ANN 10. Each of the neurons in the first layer 103-1 performs a MAC operation between the input vector 102 and the corresponding weights in the matrix 104. An activation function is applied to the partial results, and an output vector is generated, which is provided as an input vector to the next layer, i.e., the second layer 103-2. This process occurs for each neuron in each layer. For example, the last layer 103-L performs a MAC operation between the output vector of the previous layer and the corresponding weights in the matrix 104. An activation function is applied to the partial results, and an output vector 106 is generated. In many embodiments, the output layer can follow layer 103-L.

[0033] CNN can be used for classification or recognition applications, such as image recognition, speech recognition, etc. CNN has an input layer, an output layer, and multiple hidden layers, including convolutional layers, pooling layers, normalization layers, fully connected layers, etc. Each convolutional layer applies a sliding dot product or a cross correlation with the input volume, applies an activation function to the result, and then provides the activation or output volume to the next layer. The convolutional layer typically uses the ReLU function as an activation function. In some embodiments, the activation function is provided in a separate activation layer (such as a ReLU layer). The pooling layer reduces the size of the output volume received from the previous convolutional layer, and can calculate the average or maximum value of small cluster data, such as a 2×2 matrix. In some embodiments, the convolutional layer and the pooling layer can form a single layer of the CNN. The fully connected layer follows the convolutional layer and the pooling layer, and includes a flat layer and a classification layer, followed by a normalization layer including a normalization function (such as a SoftMax function). The output layer follows the last fully connected layer; in some embodiments, the output layer may include a normalization function.

[0034] Figure 2A A CNN 15 is depicted according to an embodiment of the present disclosure.

[0035] CNN 15 includes: an input layer 20; one or more hidden layers, such as convolutional layer 30-1, pooling layer 30-2, hidden (flat) layer 40, hidden (classification) layer 50, etc.; and an output layer 60. Many other variations of input layers, hidden layers, and output layers are contemplated.

[0036] The input layer 20 includes one or more input nodes 21, etc., which present input data (such as a color image) as an input volume to the first convolutional layer, such as convolutional layer 30-1. The input volume is a three-dimensional matrix with width, height and depth. For example, the input data representing a color image is presented as an input volume of 512 pixels×512 pixels×3 channels (red, green, blue); other input volume sizes may also be used, such as 32×32×3, 64×64×3, 128×128×3, etc., 32×32×1, 64×64×1, 128×128×1, 512×512×1, etc.

[0037] The convolutional layer 30-1 is locally connected to the input layer 20 and includes a plurality of nodes (not shown for clarity) connected to local regions in the input volume. For a CNN using standard convolution, each node computes the dot product between the node's weight and the corresponding local region of the input volume. An activation function is then applied to the result of each convolutional calculation to produce an output volume that is provided as an input volume to subsequent layers. The activation function can be applied by each convolutional layer node or by the nodes of a subsequent locally connected ReLU layer.

[0038] The pooling layer 30-2 is locally connected to the convolutional layer 30-1 and includes multiple nodes (not shown for clarity) connected to local areas in the input volume. The pooling layer 30-2 also produces an output volume that is provided as an input volume to subsequent layers, such as another convolutional layer 30-1, a flat layer 40, etc. In some embodiments, the convolutional layer 30-1 and the pooling layer 30-2 form a single hidden layer 30. Similarly, in some embodiments, the convolutional layer 30-1, the ReLU layer, and the pooling layer 30-2 form a single hidden layer 30. In general, the output volumes of the convolutional and pooling layers can be described as feature maps, and one or more single hidden layers 30 form the feature learning portion of the CNN 15.

[0039] The hidden layer 40 is a "flattening" layer that is locally connected to the pooling layer 30-2 and includes one or more hidden (flattening) nodes 41, 42, 43, 44, 45, etc. The hidden (flattening) layer 40 "flattens" the output volume produced by the previous pooling layer 30-2 into a column vector provided to the subsequent, fully connected hidden layer 50.

[0040] The hidden layer 50 is a classification layer that is fully connected to the hidden (flat) layer 40 and includes one or more hidden (classification) nodes 51, 52, 53, 54, 55, etc.

[0041] The output layer 60 includes one or more output nodes 61, 62, etc., and is fully connected to the hidden (classification) layer 50. The fully connected output layer 60 receives the classification result output through the hidden (classification) layer 50, and each node outputs a predicted class score. A normalization function (such as a Softmax function) may be applied to the predicted class score through the output layer 60 or alternatively through an additional layer inserted between the hidden (classification) layer 50 and the output layer 60.

[0042] Figure 2B A convolution operation 200 is depicted within a convolutional layer 30 of a CNN 15 according to an embodiment of the present disclosure. The convolutional layer typically includes "M" filters, "C" input channels, "C" input feature maps (i.e., one input feature map for each input channel), and "M" output feature maps (i.e., one output feature map for each filter). Each filter has "C" sets of weights (i.e., each filter has a set of weights for each input channel), and convolution is performed across the input feature maps to produce an output feature map corresponding to the filter. Convolutional layers typically require the movement of a large amount of data, generate a significant computational load, and require a buffer of considerable size to store intermediate values.

[0043] More specifically, the convolution operation 200 includes: M filters 2021, 2022, ..., 202 M; Input feature map 204, including "C" input data matrices 2041, 2042, ..., 204 N (ie, N is equal to C); and output feature map 206, including "M" output data matrices 2061, 2062, ..., 206 M For illustration purposes, the weight set within each filter includes a 2×2 weight matrix, each input feature map includes a 6×6 input data matrix, and each output feature map includes a 5×5 output data matrix. Figure 2B For the specific arrangement shown in , the total number of operations is 2×2×N×5×5×M.

[0044] Figure 2C Another convolutional layer operation 201 of the CNN 15 according to an embodiment of the present disclosure is depicted. The convolutional layer operation 201 is a simplified version of the convolutional layer operation 200, including three input channels, namely "C" and "N" equal to 3, and a single filter 2021, namely "M" equal to 1.

[0045] The input feature map 204 (6×6×3) includes an input data matrix 2041, an input data matrix 2042, and an input data matrix 2043, and the filter 2021 (2×2×3) includes a weight matrix 202 1.1 (w 1 ), weight matrix 202 1.2 (w 2 ) and weight matrix 202 1.3 (w 3 ), and the output data matrix 2061 (5×5×1) includes the output data matrix. Filter 2021 is convolved with the input feature map 204 to produce an output feature map 206 having a single output data matrix 2061. In this example, the output data matrix element o1 is the filter 202 1.1 (w 1 ) and the upper left quadrant 2041 of the input data matrix (a 1 q1 ), filter 202 1.2 (w 2 ) and the upper left quadrant 2042 of the input data matrix (a 2 q1 ) and filter 202 1.3 (w 3 ) and the upper left quadrant 2043 of the input data matrix (a 3 q1 ) is the sum of the dot products of .

[0046] More specifically, filter 202 1.1 (w 1 ) and the upper left quadrant 2041 of the input data matrix (a 1q1 ) is equal to w 1 1.a 1 1+w 1 2.a 1 2+w 1 3.a 1 7+w 1 4.a 1 8. Calculate filter 202 in the same way 1.2 (w 2 ) and the upper left quadrant 2042 of the input data matrix (a 2 q1 ) and filter 202 1.3 (w 3 ) and the upper left quadrant 2043 of the input data matrix (a 3 q1 ), that is, filter 202 1.2 (w 2 ) and the upper left quadrant 2042 of the input data matrix (a 2 q1 ) is equal to w 2 1.a 2 1+w 2 2.a 2 2+w 2 3.a 2 7+w 2 4.a 2 8, and filter 202 1.3 (w 3 ) and the upper left quadrant 2043 of the input data matrix (a 3 q1 ) is equal to w 3 1.a 3 1+w 3 2.a 3 2+w 3 3.a 3 7+w 3 4.a 3 8.

[0047] The output data matrix element o2 is the filter 202 1.1 (w 1 ) and the next upper quadrant 2041 of the input data matrix, the filter 202 1.2 (w 2 ) and the next upper quadrant 2042 of the input data matrix and the filter 202 1.3 (w 3) and the sum of the dot products of the next upper quadrant 2043 of the input data matrix. The "next" upper quadrant 2041, 2042 and 2043 in each input data matrix has been shifted one column to the right relative to the first upper quadrant.

[0048] More specifically, filter 202 1.1 (w 1 ) and the dot product of the next upper quadrant 2041 of the input data matrix is ​​equal to w 1 1.a 1 2+w 1 2.a 1 3+w 1 3.a 1 8+w 1 4.a 1 9. Calculate filter 202 in the same way 1.2 (w 2 ) and the dot product of the next upper left quadrant 2042 of the input data matrix and the filter 202 1.3 (w 3 ) and the dot product of the next upper left quadrant 2043 of the input data matrix, i.e., the filter 202 1.2 (w 2 ) and the upper left quadrant 2042 of the input data matrix (a 2 q1 ) is equal to w 2 1.a 2 2+w 2 2.a 2 3+w 2 3.a 2 8+w 2 4.a 2 9, and filter 202 1.3 (w 3 ) and the upper left quadrant 2043 of the input data matrix (a 3 q1 ) is equal to w 3 1.a 3 2+w 3 2.a 3 3+w 3 3.a 3 8+w 3 4.a 3 9.

[0049] The remaining output data matrix elements o3 to o5 are calculated in a similar manner. After the output data matrix element o5 has been calculated, the "next" quadrant 2041, 2042 and 2043 in each input data matrix is ​​shifted all the way to the leftmost column and shifted down one row to be used to calculate the output data matrix element o6. Then, the output data matrix elements o7 to o8 are calculated by shifting the quadrants in each input data matrix 2041, 2042 and 2043 to the right by one column. 10 The remaining output data matrix elements o are calculated in a similar manner 11 to 15 , o 16 to 20 and 21 to 25 .

[0050] Generally speaking, local convolution operations are not performed by CNNs due to the complex data flows and expensive data paths that are typically required. Instead, local convolution operations are converted into general matrix multiplication (GEMM) operations, and then the GEMM operations are more efficiently implemented using optimized software libraries or dedicated hardware through a central processing unit (CPU), a dedicated processor, a hardware accelerator processing engine, etc. More specifically, the "IM2COL" software function can be used to convert the filter (weight) matrix and the input feature map (IFM) matrix of each convolution operation into an extended format compatible with the GEMM operation. An IM2COL version of each filter (weight) matrix and each IFM matrix is ​​generated and stored in memory, and then loaded from memory and processed by the GEMM operation.

[0051] Figure 2D Depicted are the transformed convolutional layer operations of a CNN according to an embodiment of the present disclosure.

[0052] In one embodiment, the convolution layer operation of the CNN implemented on a central processing unit (CPU) can be converted into a general matrix multiplication (GEMM) operation. The convolution layer operation 201 is converted into a GEMM operation by converting the filter 2021 into a converted weight matrix 212 (1×12) and converting the input feature map 204 into a converted input data matrix 214 (12×25). After multiplying the converted weight matrix 212 and the converted input data matrix 214, the converted output data matrix 216 (1×25) is then reshaped into an output data matrix 2061 (5×5). For ease of explanation, the converted input data matrix 214 is depicted in abbreviated form.

[0053] In this example, the transformed output data matrix element o1 is the sum of the dot products of the first (ie, only one) row of the transformed weight matrix 212 and the first column of the transformed input data matrix 214. Figure 1B As shown in FIG. 2 , the transformed weight matrix 212 includes the filter 202 1.1 (w 1 ), filter 202 1.2 (w 2 ) and filter 202 1.3 (w 3 ), and the first row of the transformed input data matrix 214 includes the upper left quadrant 2041 (a 1 q1 ), the upper left quadrant 2042 of the input data matrix (a 2 q1 ) and the upper left quadrant 2043 of the input data matrix (a 3 q1 ) elements.

[0054] More specifically, the transformed output data matrix element o1 is equal to w 1 1.a 1 1+w 1 2.a 1 2+w 1 3.a 1 7+w 1 4.a 1 8+w 2 1.a 2 1+w 2 2.a 2 2+w 2 3.a 2 7+w 2 4.a 2 8+w 3 1.a 3 1+w 3 2.a 3 2+w 3 3.a 3 7+w 3 4.a 3 8. As shown above, the transformed output data matrix element o1 is equal to the output data matrix element o1.

[0055] Figure 3A FIG. 1 shows an implementation of a NVM crossbar switch 301 according to an embodiment of the present disclosure. Figure 2D Part of the convolutional layer operation.

[0056] The NVM crossbar switch 301 includes twelve row signal lines 308, namely row signal lines 3081, . . . , 308 12and a single column signal line 312. A bit cell is disposed at each intersection of the row signal line 308 and the column signal line 312. Each bit cell includes one or more NVM elements that are programmed to represent a particular weight associated with the filter 2021, i.e., w 1 1. w 1 2 is proportional to the conductivity value. The conductivity is calculated by g 1 1. g 1 2. g 1 3. g 1 4. g 2 1. g 2 2. g 2 3. g 2 4. g 3 1. g 3 2. g 3 3 and g 3 4 indicates that they are respectively related to weight w 1 1. w 1 2. w 1 3. w 1 4. w 2 1. w 2 2. w 2 3. w 2 4. w 3 1. w 3 2. w 3 3 and w 3 4 in proportion.

[0057] To calculate the first element, o1, of the transformed output data matrix 216, the input data from the first column of the transformed input data matrix 214 is provided to the crossbar 302 as a plurality of rows along the row signal lines 3081, . . . , 308 12 The input a 1 1.a 1 2.a 1 7.a 1 8.a 2 1.a 2 2.a 2 7.a 2 8.a 3 1.a 3 2.a 3 7 and a 3 8. The input data is converted into an analog voltage v by a digital-to-analog converter (DAC) 310 1 1. v 1 2. v 1 7. v 1 8. v 2 1. v 2 2. v 2 7. v 2 8. v3 1. v 3 2. v 3 7 and v 3 8 and provided to the column signal line 312. The column signal line 312 carries a bit line (BL) signal that is proportional to the accumulated dot product of the input data and the weights along the column signal line 312. The BL signal is then digitized using an analog-to-digital converter (ADC); bias, scaling, and activation functions may be applied to the digitized signal to obtain the first element of the transformed output data matrix 216, i.e., o1.

[0058] Figure 3B Depicted is an architecture 300 for convolutional layer operations implemented using an NVM crossbar 302 in accordance with an embodiment of the present disclosure.

[0059] The NVM crossbar switch 302 includes row signal lines 308, namely row signal lines 3081, 3082, ..., 308 L , and column signal lines 312, namely column signal lines 3121, 3122, ..., 312 M The bit cell 314 is arranged at each intersection of the row signal line 308 and the column signal line 312, that is, the bit cell 314 11 , …, 314 LM Each bit cell 314 includes one or more NVM elements that are programmed to represent a particular weight associated with a particular filter 202 (e.g., W for filter 2021). 0 11 In the first column, i.e., column signal line 3121, the conductance is depicted as G 0 11 , G 0 12 , …, G C 21 , G C 22 .

[0060] Due to the low resistance state (LRS) R 开 With high resistance state (HRS) R 关 Due to the wide separation between the two bits, it is often useful to encode multiple linearly separated resistance levels within individual bit cells. In the case of correlated electronic material RAM (CeRAM), the ratio of HRS / LRS is at least 2 orders of magnitude, which supports 4-bit encoding (ie, 16 resistance levels).

[0061] Data from the input feature map 204 is provided to the crossbar 302 as data along row signal lines 3081, ..., 308 L Input I 0 11 ,…,I C22 , which is converted into an analog voltage V by a digital-to-analog converter (DAC) 0 11 ,…,V C 22 , and is transmitted across the NVM cell. In this embodiment, L is equal to four times the number of input channels, that is, 2×2×C. The DAC includes DAC elements 3101, ..., 310 L Column signal lines 3121, ..., 312 M Transmits the corresponding bit line (BL) signal BL 1 ,…,BL M , each of which is proportional to the accumulated dot product of the input data and the weight along the column signal line. Each BL signal is then digitized using an analog-to-digital converter (ADC); a bias function, a scaling function, and an activation function may be applied to the digitized signal to obtain output data of the output feature map 206.

[0062] exist Figure 3B , a convolutional layer operation is generally depicted as mapping to an NVM crossbar having M filters and C input channels. In some embodiments, the weights are static, i.e., the weights are programmed into the NVM crossbar once and do not change during the course of an inference operation. Typical NVM elements (such as phase change memory and resistive RAM) have a limited write "endurance," and may be able to write to these NVM elements a limited number of times (e.g., approximately 10 8 Other NVM elements, such as magnetic RAM and CeRAM, can exhibit relatively high endurance (approximately 10 12 ), but continuous operation may incur a limited lifetime. Such lifetime limitations therefore place significant constraints on accelerator architectures that rely on updating weights with every inference cycle.

[0063] For example, for an IoT class accelerator operating at 100 MHz, a 8 An accelerator with a durability of 10 12 The endurance of an accelerator may have a lifespan of 10,000 seconds or 4 days at peak usage. Unfortunately, such weights may not be streamed from external DRAM and must be fixed on-chip. Furthermore, NVM bit cells suffer from high write power and the attendant expensive power consumption, which may affect the effective performance of update operations. As a result, the write phase may be problematic and take a long time to complete.

[0064] Such an arrangement behaves differently from SRAM, which has significantly higher write endurance, and may not be suitable for reprogramming weights during inference. Therefore, the entire ANN may need to be expanded into an on-chip crossbar and fixed during inference. Although this option has the advantage of eliminating DRAM power consumption, it may undesirably limit the maximum size of the ANN that can be programmed on chip. In addition, this option also causes area loss because mapping larger ANNs requires the instantiation of a crossbar with a capacity of megabits. Due to yield loss, this consumes a higher area and makes it more prone to chip failure. In addition, instantiating multiple crossbars requires the instantiation of multiple ADCs / DACs, all of which need to be programmed, trimmed, and drift compensated.

[0065] NVM / CeRAM elements are a specific type of random access memory formed in whole or in part from correlated electronic materials. CeRAM can exhibit abrupt conductive or insulating state transitions caused by electronic correlation rather than solid-state structural phase changes (such as caused by filament formation and conduction in resistive RAM devices). In contrast to melting / solidification or filament formation, the abrupt conductor / insulator transitions in CeRAM can be in response to quantum mechanical phenomena.

[0066] The quantum mechanical transition of CeRAM between the insulating state and the conducting state can be understood in terms of the Mott transition. In the Mott transition, the material can switch from the insulating state to the conducting state if the Mott transition conditions are generated. When the critical carrier concentration is achieved so that the Mott criterion is met, the Mott transition will occur and the state will change from high resistance / impedance (or capacitance) to low resistance / impedance (or capacitance).

[0067] The "state" or "memory state" of a CeRAM element may depend on the impedance state or the conductive state of the CeRAM element. In this context, "state" or "memory state" means a detectable state of a memory device that indicates a value, symbol, parameter, or condition, to provide just a few examples. In a particular implementation, the memory state of a memory device may be detected based at least in part on signals detected at terminals of the memory device in a read operation. In another implementation, a memory device may be in a particular memory state to represent or store a particular value, symbol, or parameter by applying one or more signals across the terminals of the memory device in a "write operation."

[0068] The CeRAM element may include a material sandwiched between conductive terminals. The material may be transformed between the aforementioned conductive state and insulating state by applying a specific voltage and current between the terminals. The material of the CeRAM element sandwiched between the conductive terminals may be in an insulating state by applying a first programming signal across the terminals having a reset voltage and a reset current at a reset current density, or may be in a conductive state by applying a second programming signal across the terminals having a set voltage and a set current at a set current density.

[0069] The accumulated dot product of the input data and the weight along each column signal line 312 with respect to the NVM crossbar switches 301 and 302 can be represented by Equation 2, which represents a simple MAC operation. i ) multiplied by the appropriate weight (i.e., the conductance G i ), and the products are accumulated into the result (ie, the current along BL).

[0070] BL=∑ i＝0..L (G i *Vi) (Equation 2)

[0071] The embodiments of the present disclosure advantageously reconfigure the MAC operation of each column signal line 312 into a voltage V i The initial sum is multiplied by the specific conductance value G k , followed by the product of the sum of each voltage and the specific conductance value G k In some embodiments, the conductivity value can be represented by a 4-bit (K-bit) number that provides 16 different values ​​(K values), such as G1, G2, G3, G4, G5, G6, G7, G8, G9, G10, G11, G12, G13, G14, G15, G16, G17, G18, G19, G20, G21, G22, G23, G24, G25, G26, G27, G28, G29, G30, G31, G32, G33, G34, G35, G36, G37, G38, G39, G40, G41, G42, G43, G44, G45, G46, G47, G48, G49, G50, G51, G52, G53, G54, G55, G56, G57, G 10 , G 11 , G 12 , G 13 , G 14 , G 15 and G 16 Other bit sizes are also contemplated, such as 2 bits, 3 bits, etc.

[0072] For example, relative to Figure 3A , conductivity g 1 1 can have the value G1, the conductance g 1 2 can have the value G2, the conductance g 1 3 can have the value G3, the conductance g 1 4 can have the value G4, the conductance g 2 1 can have a value of G3, the conductance g 2 2 can have a value of G4, the conductance g 2 3 can have a value of G5, the conductance g 2 4 can have a value of G6, the conductance g3 1 can have a value of G5, the conductance g 3 2 can have a value of G6, the conductance g 3 3 can have a value of G7, and the conductance g 3 4 can have the value G8.

[0073] In this example, the conductance g 1 3 and g 2 1 has the same conductance G3, conductance g 1 4 and g 2 2 have the same conductance G4, conductance g 2 3 and g 3 1 has the same conductivity G5, conductivity g 2 4 and g 3 2 have the same conductance G6. Therefore, we can first 1 7 and V 2 1, and then multiplied by the conductance G3 during the subsequent MAC operation, V 1 8 and V 2 2 and then multiplied by the conductance G4 during the subsequent MAC operation, V 2 7 and V 3 1, and then multiplied by the conductance G5 during the subsequent MAC operation, V 2 8 and V 3 2 is summed and then multiplied by conductance G6 during a subsequent MAC operation.

[0074] Alternatively, the remaining voltages may be set to zero (0) and then all voltages may be summed initially, rather than just the voltages to be multiplied by a particular conductance value G. k The voltage V i In the above example, for conductance G3, another 10 voltages are set to zero, i.e., V 1 1. V 1 2. V 1 8. V 2 2. V 2 7. V 2 8. V 3 1. V 3 2. V 3 7 and V 3 8, and then sum all the voltages. In general, this process is repeated for each conductance value, which includes the conductance to be multiplied by a single voltage. In the above example, for conductance G1, another 11 voltages are set to zero, i.e., V 1 2. V 1 7. V 1 8. V 2 1. V 2 2. V2 7. V 2 8. V 3 1. V 3 2. V 3 7 and V 3 8, and then sum all voltages.

[0075] The reconstructed MAC operation of each column signal line 312 can be expressed by Equation 3 and Equation 4.

[0076] BL=G0*∑ i＝0..L V i +G1*∑ i＝0..L V i +…+G 15 *∑ i＝0..L V i (Equation 3)

[0077] BL=∑ k＝0..15 G k *∑ i＝0..L V i (Equation 4)

[0078] where, for each input data value L, when G k does not correspond to the appropriate weight for the input data value, V i is set to zero (0).

[0079] Advantageously, as shown by Equation 4, refactoring the MAC operation results in a simpler implementation, where all input multipliers are initially added to conditionally add the input activations together, depending on whether they are included in the MAC operation with a specific weight value. NVM elements may be used to complete the initial addition operation. However, according to embodiments of the present disclosure, these NVM elements do not need to be precisely programmed. Binary Weight Coding (R 开 / R 关 ) is exploited to connect input activations to weight values ​​without precise programming.

[0080] Figure 4A A schematic diagram of an architecture 401 of a reconfigured MAC operation according to an embodiment of the present disclosure is depicted. In this embodiment, the architecture 401 includes a DAC block 402, a crossbar switch 404, and a multiplier / scaling module 406.

[0081] Typically, the crossbar switch 404 includes M×N elements. In this embodiment, M is equal to 12, N is equal to 16, and the crossbar switch 404 includes 192 elements, nodes, or bit cells 408. 1,1 , …, 408 12,16 , provided on 12 row signal lines 4101, ..., 410 12and 16 column signal lines 4121, ..., 412 16 Each corresponding bit cell 408 1,1 , …, 408 12,16 Includes one or more NVM elements, consisting of s i j Indicates that the state of the bit unit is stored (for example, R 开 / R 关 ). The bit cell 408 is capable of switching between a first impedance state and a second impedance state (R 开 / R 关 ).

[0082] The multiplier / scaling module 406 includes 16 bit cells 4071, ..., 407 16 , the bit cell has programmable conductance or impedance g1, g2, g3, g4, g5, g6, g7, g8, g9, g 10 , g 11 , g 12 , g 13 , g 14 , g 15 and g 16 , the programmable conductance or impedance is configured to correspond to 16 different weight values ​​w1, w2, w3, w4, w5, w6, w7, w8, w9, w 10 、w 11 、w 12 、w 13 、w 14 、w 15 and w 16 .

[0083] Input data, i.e. a 1 1.a 1 2.a 1 7.a 1 8.a 2 1.a 2 2.a 2 7.a 2 8.a 3 1.a 3 2.a 3 7 and a 3 8 is provided to the DAC block 402 and converted into an analog voltage V 1 1. v 1 2. v 1 7. v 1 8. v 2 1. v 2 2. v 2 7. v 2 8. v 3 1. v3 2. v 3 7 and v 3 8, and then provided to the corresponding row signal lines 4101, ..., 410 12 By setting bit cell 408 11 , …, 408 MN Place it as R 开 / R 关 state, the voltages are first summed together in the crossbar switch 404, where the NVM elements do not need to be precisely programmed.

[0084] In the above example, the bit cell 408 associated with the voltage to be summed for a particular conductance value is set to R 开 These bit cells 408 include NVM elements s 1 1.s 2 2.s 3 3.s 4 4.s 5 3.s 6 4.s 7 5.s 8 6.s 9 5.s 10 6.s 11 7 and s 12 8(by Figure 4A The remaining NVM components are set to R 关 state, which provides a zero voltage value.

[0085] From each column signal line 4121, ..., 412 16 The summed signal is input to the corresponding bit cells 4071, ..., 407 of the multiplier / scaling module 406. 16 , and the MAC operation occurs in a single column in the multiplier / scaling module 406, where all elements are accurately programmed (16 elements total).

[0086] Typically, the converted input data matrix 214 includes the total number of input data values ​​provided to the DAC block 402, one column of the input data values ​​at a time. Figure 2D In the example shown in FIG. 1 , the total number of input data values ​​in the input data matrix is ​​300 (ie, 12×25). 1 1. ..., a 3 8) is a set of M (eg, 12) input signals selected from a total number of input signals (eg, 300) significantly greater than N (eg, 16).

[0087] Figure 4BFIG. 4 is a schematic diagram of a reconstructed MAC operation architecture 400 according to an embodiment of the present disclosure. In this embodiment, the architecture 400 includes M DACs 4031, ..., 403 M DAC block 402, cross switch 404 and multiplier / scaling module 406.

[0088] In many embodiments, the architecture 400 encodes the weights of a single filter 202 and generates a single output feature map 206 for that filter 202. In other embodiments, a separate crossbar 404 and multiplier / scaling module 406 pair may be provided for each filter 202 to generate an output feature map 206 for that filter 202. Each crossbar 404 and multiplier / scaling module 406 pair may be coupled to a common DAC block 402.

[0089] As shown, the crossbar switch 404 includes M×N elements, nodes or bit cells 408. 11 , …, 408 MN , providing row signal lines 4101, 4102, 4103, ..., 410 M and column signal lines 4121, 4122, 4123, ..., 412 N Each corresponding bit cell 408 11 , …, 408 M、N One or more NVM elements are included to store information about the state of the bit cell (e.g., R 开 / R 关 As described above, the bit cell 408 is capable of switching between a first impedance state and a second impedance state (R 开 / R 关 ).

[0090] The multiplier / scaling module 406 includes bit cells 4071, ..., 407 N , the bit cell has a programmable conductance or impedance programmed to correspond to the weights w_0, w_1, w_k-1. First, by setting the bit cell 408 11 , …, 408 MN Place it as R 开 / R 关 The state is transmitted in the crossbar switch 404, such as along the corresponding row signal lines 4101, 4102, 4103, ..., 410 M The activation a_0, a_1, ..., a_M-1, etc. of the input data are summed together, as described above, where the NVM element does not need to be precisely programmed. In this embodiment, as shown in the figure, the bit cell 408 1,1 , …, 408 MNSome of the bit cells in the 406 are switched between impedance states. The summed signal from each column N is input to the corresponding bit cell 4071, ..., 407 of the multiplier / scaling module 406. N , and the final MAC operation occurs in a single column in the multiplier / scaling module 406, where all elements are accurately programmed (N elements total).

[0091] Figure 5 A flow chart of a low-precision programming process 500 for reconstructed MAC operations is depicted in accordance with an embodiment of the present disclosure.

[0092] exist Figure 4B In the embodiment shown in FIG. 4 , the crossbar switch 404 includes M×N elements, and the multiplier / scaling module 406 includes N different weight values; K is the bit size (ie, the number of bits) of each weight value, and N=K=2 k The M×N elements are low-precision (or binary) programmed NVM elements, and the N elements are high-precision programmed NVM elements.

[0093] In block 502, the process is initialized and proceeds to block 504 of a low precision write cycle. In block 504, the index of the elements of the crossbar switch 404 is updated. In block 506, the binary resistance or conductance G of each element of the crossbar switch 404 is read. b Then, in block 508, G is written for each element of crossbar 404. b At block 510, the process terminates. In this embodiment, M×N elements are programmed to “0” (R 关 ) or "1" (R 开 ). From here, the elements of the multiplier / scaling module 406 are precisely programmed.

[0094] Figure 6 A flow chart of a high precision programming process 600 for the multiplier / scaling module 406 is depicted in accordance with an embodiment of the present disclosure.

[0095] In block 602, the process is initialized and proceeds to block 604 of a high precision write loop. In block 604, the index of the elements of the multiplier / scaling module 406 is updated. In block 606, the multi-level resistance or conductance G of each element of the multiplier / scaling module 406 is read. The high precision operation changes the conductance or resistivity to a very accurate known value. Then, in block 608, the multi-level G is tuned for each element of the multiplier / scaling module 406. Then, in block 610, the multi-level G is read, and the correct multi-level G is verified in block 612. If the multi-level G is correct, the process loops back to block 604, otherwise the process returns to block 608 for further tuning. At block 614, the process terminates.

[0096] Figure 7 A flowchart 700 of a method for performing multiply-accumulate acceleration in a neural network according to an embodiment of the present disclosure is depicted.

[0097] At block 702, the method is initialized and proceeds to block 704, which corresponds to Figure 5 The low-precision programming process 500 shown in FIG. Figure 4B , in the low precision programming process 500, the input activations a_0, a_1, ..., a_M-1 are applied to the summing array, i.e., the crossbar switch 404. The summed signal is generated by each column of the NVM elements in the summing array by the low precision programming process 500. The process proceeds to block 706, which corresponds to Figure 6 In the high-precision programming process 600 shown in FIG. 6 , the summed signal is input to a multiplication array having a plurality of NVM elements, namely, a multiplier / scaling module 406. In block 706, each NVM element in the multiplication array is precisely programmed to a resistance or conductance level proportional to the weight, such as Figure 6 At block 708, the process terminates.

[0098] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the system. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily all refer to the same embodiment.

[0099] Embodiments of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowchart illustration and / or block diagram and the combination of blocks in the flowchart illustration and / or block diagram can be implemented by computer-readable program instructions.

[0100] Some portions of the detailed description, such as the processes, may be presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. An algorithm may generally be conceived of as a set of steps leading to a desired result. These steps are steps requiring physical transformations or manipulations of physical quantities. Although not necessarily, these quantities typically take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, primarily for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0101] However, it should be taken into account that all of these and similar terms will be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically indicated by the following discussion, it should be understood that throughout the specification, discussions using terms such as "processing" or "computing" or "calculating" or "determining" or "deriving" or "displaying" refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission or display devices.

[0102] The operations described herein may be performed by a device. The device may be specially constructed for the desired purpose, or the device may include a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic card or optical card, or any type of medium suitable for storing electronic instructions. As used herein, a computer-readable storage medium should not be interpreted as being a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., a light pulse transmitted through a fiber optic cable), or an electrical signal transmitted by a wire.

[0103] Accordingly, embodiments and features of the present disclosure include, but are not limited to, the foregoing and following combinable embodiments.

[0104] In one embodiment, a device includes a crossbar switch and a multiplier. The crossbar switch includes a plurality of crossbar switch nodes arranged in an array of rows and columns, each crossbar switch node is programmable to a first resistance level or a second resistance level, and the crossbar switch is configured to sum a plurality of analog input activation signals on each column of the crossbar switch node and output a plurality of summed activation signals. The multiplier is coupled to the crossbar switch and includes a plurality of multiplier nodes, each multiplier node is programmable to a different conductance level proportional to one of a plurality of neural network weights, and the multiplier is configured to sum the plurality of summed activation signals on the multiplier node and output an analog output activation signal.

[0105] In another embodiment of the apparatus, each crossbar node includes one or more non-volatile elements (NVMs), and each multiplier node includes a plurality of NVMs.

[0106] In another embodiment of the apparatus, the crossbar includes M rows, N columns, and M×N crossbar nodes, and the multiplier includes N multiplier nodes.

[0107] In another embodiment of the device, the device further includes a plurality of digital-to-analog converters (DACs) coupled to the crossbar switch, each DAC being configured to receive the digital input activation signal and output one of a plurality of analog input activation signals.

[0108] In another embodiment of the apparatus, each non-volatile memory element in each multiplier node is programmed to a different conductance level based on the bit size k of the neural network weights.

[0109] In another embodiment of the device, the total number of conductance levels is equal to 2 k .

[0110] In one embodiment, a method includes generating a summed signal for each column in a summing array including a plurality of nonvolatile memory elements arranged in columns based at least in part on a plurality of input signals, each nonvolatile memory element in the summing array being programmed to a high resistance state or a low resistance state based on weights of a neural network; and generating an output signal in a multiplying array including a plurality of nonvolatile memory elements based at least in part on the summed signal from the summing array, each nonvolatile memory element in the multiplying array being programmed to a different conductance level based on weights of the neural network.

[0111] In another embodiment of the method, each nonvolatile memory element in the summing array is configured to receive an input signal and add the input signal to the summed signal of the corresponding column based on the programmed resistance state; each nonvolatile memory element in the multiplying array is configured to receive the summed signal from a different column of the summing array and adjust the summed signal based on the programmed conductance level to generate an adjusted summed signal; and the output signal is a combination of the adjusted summed signals.

[0112] In another embodiment of the method, each nonvolatile memory element in the summing array is configured to add an input signal to the summed signal of the corresponding column when programmed to a low resistance state, and not to add the input signal to the summed signal of the corresponding column when programmed to a high resistance state.

[0113] In another embodiment of the method, the summation array is a crossbar having M rows, N columns, and M x N non-volatile memory elements, and wherein the multiplication array has N elements.

[0114] In another embodiment of the method, the plurality of input signals is a set of M input signals selected from a total number of input signals significantly greater than N.

[0115] In another embodiment of the method, each non-volatile memory element in the multiplication array is programmed to a different conductance level based on the bit size k of the weights of the neural network.

[0116] In another embodiment of the method, the total number of conductivity levels is equal to 2 k .

[0117] In one embodiment, another device includes a summing array, the summing array including a plurality of nonvolatile memory elements arranged in columns, each nonvolatile memory element in the summing array being programmed to a high resistance state or a low resistance state based on a weight of a neural network, the summing array being configured to generate a summed signal for each column based at least in part on a plurality of input signals; and a multiplication array coupled to the summing array, the summing array including a plurality of nonvolatile memory elements, each nonvolatile memory element in the multiplication array being programmed to a different conductance level based on the weight of the neural network, the multiplication array being configured to generate an output signal based at least in part on the summed signal from the summing array.

[0118] In another embodiment of the device, each nonvolatile memory element in the summing array is configured to receive an input signal and add the input signal to the summed signal of the corresponding column based on the programmed resistance state; each nonvolatile memory element in the multiplying array is configured to receive the summed signal from a different column of the summing array and adjust the summed signal based on the programmed conductance level to generate an adjusted summed signal; and the output signal is a combination of the adjusted summed signals.

[0119] In another embodiment of the device, each non-volatile memory element in the summing array is configured to add the input signal to the summed signal of the corresponding column when programmed to a low resistance state, and not to add the input signal to the summed signal of the corresponding column when programmed to a high resistance state.

[0120] In another embodiment of the apparatus, the summation array is a crossbar having M rows, N columns, and M x N non-volatile memory elements, and wherein the multiplication array has N elements.

[0121] In another embodiment of the apparatus, the plurality of input signals is a set of M input signals selected from a total number of input signals significantly greater than N.

[0122] In another embodiment of the apparatus, each non-volatile memory element in the multiplication array is programmed to a different conductance level based on the bit size k of the weights of the neural network.

[0123] In another embodiment of the device, the total number of conductance levels is equal to 2 k .

[0124] It should be understood that the examples and corresponding figures used herein are for illustrative purposes only. Different configurations and terms may be used without departing from the principles expressed herein. For example, components and modules may be added, deleted, modified or arranged in different connections without departing from these principles.

[0125] In the following detailed description of the embodiments of the present disclosure, numerous specific details are set forth in order to provide a more thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other cases, well-known features are not described in detail to avoid unnecessarily complicating this specification.

[0126] It should be understood that the terms used herein are for the purpose of describing various embodiments according to the present disclosure, and are not intended to be restrictive. As used herein, the term "a / an" is defined as one or more than one. As used herein, the term "multiple" is defined as two or more than two. As used herein, the term "another" is defined as at least a second or more. As used herein, the terms "include" and / or "have" are defined as including (i.e., open language). As used herein, the term "coupled" is defined as connected, but not necessarily directly and not necessarily mechanically connected. The term "provided" is defined in this article with its broadest meaning, for example, as a whole or simultaneously or within a certain period of time, multiple parts begin to become / become physical existence, become available and / or supply to someone or something.

[0127] As used herein, the terms "about" and "approximately" apply to all numerical values, whether or not such values ​​are explicitly indicated. Such terms generally refer to a range of numbers that one skilled in the art would consider equivalent to the stated value (i.e., having the same function or result). These terms may include numbers rounded to the nearest significant figure. In this document, any reference to the term "longitudinal" should be understood to mean in a direction corresponding to the direction of elongation of a personal computing device from one terminal end to the opposite terminal end.

[0128] According to the foregoing, a method and architecture for performing multiply-accumulate acceleration is disclosed. Thus, having described the disclosure of the present application in detail and with reference to the embodiments of the present disclosure, it will be apparent that modifications and variations are possible without departing from the scope defined in the following appended claims.

Claims

1. A device, comprising: a crossbar switch including a plurality of crossbar nodes arranged in an array of rows and columns, each crossbar node being programmable to a first resistance level or a second resistance level, the crossbar switch being configured to sum a plurality of analog input activation signals at each column of the crossbar nodes and output a plurality of summed activation signals; and a multiplier coupled to the crossbar, the multiplier comprising a plurality of multiplier nodes, each multiplier node programmable to a different conductance level proportional to one of a plurality of neural network weights, the multiplier configured to sum the plurality of summed activation signals at the multiplier nodes and output an analog output activation signal.

2. The apparatus of claim 1, wherein each crossbar switch node comprises one or more non-volatile elements (NVMs), and each multiplier node comprises a plurality of non-volatile elements (NVMs). 3 . The apparatus of claim 2 , wherein the crossbar comprises M rows, N columns, and M×N crossbar nodes, and the multiplier comprises N multiplier nodes.

4. The device according to claim 1, further comprising: A plurality of digital-to-analog converters (DACs) are coupled to the crossbar switch, each of the DACs being configured to receive a digital input activation signal and output one of the plurality of analog input activation signals.

5. The apparatus of claim 1 , wherein each non-volatile memory element in each multiplier node is programmed to the different conductance level based on a bit size k of the neural network weights.

6. The apparatus of claim 5, wherein the total number of conductance levels is equal to 2 k .

7. A method for performing a reconstructed multiplication and accumulation operation, the method comprising: generating a summed signal for each column based at least in part on a plurality of input signals in a summing array including a plurality of nonvolatile memory elements arranged in columns, each nonvolatile memory element in the summing array being programmed to a high resistance state or a low resistance state based on a weight of a neural network; as well as An output signal is generated based at least in part on the summed signal from the summing array in a multiplying array comprising a plurality of nonvolatile memory elements, each nonvolatile memory element in the multiplying array being programmed to a different conductance level based on the weights of the neural network.

8. The method according to claim 7, wherein: each nonvolatile memory element in the summing array being configured to receive an input signal and to add the input signal to the summed signal of a corresponding column based on the programmed resistance state; each nonvolatile memory element in the multiplying array is configured to receive a summed signal from a different column of the summing array and adjust the summed signal based on the programmed conductance level to generate an adjusted summed signal; and The output signal is a combination of the adjusted summed signals.

9. The method of claim 8, wherein each nonvolatile memory element in the summing array is configured to add the input signal to the summed signal of the corresponding column when programmed to the low resistance state, and not to add the input signal to the summed signal of the corresponding column when programmed to the high resistance state.

10. The method of claim 9, wherein the summation array is a crossbar having M rows, N columns, and M x N non-volatile memory elements, and wherein the multiplication array has N elements.

11. The method of claim 10, wherein the plurality of input signals is a set of M input signals selected from a total number of input signals significantly greater than N.

12. The method of claim 7, wherein each non-volatile memory element in the multiplication array is programmed to the different conductance level based on a bit size k of the weights of the neural network.

13. The method of claim 12, wherein the total number of conductivity levels is equal to 2 k .

14. A device, comprising: a summing array comprising a plurality of nonvolatile memory elements arranged in columns, each nonvolatile memory element in the summing array being programmed to a high resistance state or a low resistance state based on a weight of the neural network, the summing array being configured to generate a summed signal for each column based at least in part on a plurality of input signals; and a multiplication array coupled to the summing array, the multiplication array comprising a plurality of non-volatile memory elements, each non-volatile memory element in the multiplication array being programmed to a different conductance level based on the weights of the neural network, the multiplication array being configured to generate an output signal based at least in part on the summed signal from the summing array.

15. The apparatus of claim 14, wherein: each nonvolatile memory element in the summing array being configured to receive an input signal and to add the input signal to the summed signal of a corresponding column based on the programmed resistance state; each nonvolatile memory element in the multiplying array is configured to receive a summed signal from a different column of the summing array and adjust the summed signal based on the programmed conductance level to generate an adjusted summed signal; and The output signal is a combination of the adjusted summed signals.

16. The apparatus of claim 15 , wherein each nonvolatile memory element in the summing array is configured to add the input signal to the summed signal of the corresponding column when programmed to the low resistance state, and not to add the input signal to the summed signal of the corresponding column when programmed to the high resistance state.

17. The apparatus of claim 16, wherein the summation array is a crossbar having M rows, N columns, and M x N non-volatile memory elements, and wherein the multiplication array has N elements.

18. The apparatus of claim 17, wherein the plurality of input signals is a set of M input signals selected from a total number of input signals significantly greater than N.

19. The apparatus of claim 14, wherein each non-volatile memory element in the multiplication array is programmed to the different conductance level based on a bit size k of the weights of the neural network.

20. The apparatus of claim 19, wherein the total number of conductance levels is equal to 2 k .

Images