Parallel processing of convolutional layers of neural networks using in-memory compute arrays

Abstract

A device includes first and second compute-in-memory (CIM) arrays. The first CIM array is configured to store weights corresponding to a filter tensor, receive a first set of activations corresponding to a first receptive field of an input, and process the first set of activations using the weights to generate a corresponding first tensor of output values. The second CIM array is configured to store a first copy of the weights corresponding to the filter tensor and receive a second set of activations corresponding to a second receptive field of an input. The second CIM array is further configured to process the second set of activations using the first copy of the weights to generate a corresponding second tensor of output values. The first and second compute-in-memory arrays are configured to process the first and second receptive fields in parallel.

Description

[0001] Priority claim under 35 U.S.C. § 119

[0002] This patent application claims priority to non-provisional application No. 16 / 576,597, filed on September 19, 2019, entitled “PARALLEL PROCESSING OF A CONVOLUTIONAL LAYER OF A NEURAL NETWORK WITH COMPUTE-IN-MEMORY ARRAY,” which is assigned to the assignee of the present application and is hereby expressly incorporated herein by reference.

[0003] public domain

[0004] Aspects of the present disclosure generally relate to parallel processing of convolutional layers of a neural network with an in-memory compute array. Background Art

[0005] Conventionally, convolutional layers in convolutional neural networks (CNNs) use small filters. However, CNNs sometimes have a very large number of input and output channels. Therefore, filters are reused multiple times in multiply-and-accumulate (MAC) operations to generate all outputs, which takes many cycles and is time-consuming.

[0006] In some digital implementations (e.g., 32-bit or less), filters of a convolutional neural network can be digitally cloned. Although digital cloning filters reduce the number of cycles in a convolutional neural network, they significantly increase power consumption. This is because the digital cloning process involves moving filter weights and parallel activations, which consumes a lot of energy in a short period of time.

[0007] Overview

[0008] A device includes first and second compute-in-memory (CIM) arrays. The first CIM array is configured to store a plurality of weights corresponding to a filter tensor, receive a first set of activations corresponding to a first receptive field of an input, and process the first set of activations using the weights to generate a corresponding first tensor of output values. The second CIM array is configured to store a first copy of the weights corresponding to the filter tensor, receive a second set of activations corresponding to a second receptive field of the input, and process the second set of activations using the first copy of the weights to generate a corresponding second tensor of output values. The first compute-in-memory array and the second compute-in-memory array are configured to process the first receptive field and the second receptive field in parallel.

[0009] A method for processing a convolutional layer of a convolutional neural network includes storing a plurality of weights corresponding to a filter tensor in a first in-memory computation array. The method further includes receiving, by the first in-memory computation array, a first set of activations corresponding to a first receptive field of an input. The method includes storing, by the second in-memory computation array, a first copy of the weights corresponding to the filter tensor. The method further includes receiving, by the second in-memory computation array, a second set of activations corresponding to a second receptive field of the input. In addition, the method includes processing the first set of activations using the weights in the first in-memory computation array to generate a corresponding first tensor of output values, and processing the second set of activations using the first copy of the weights in the second in-memory computation array to generate a corresponding second tensor of output values. The first in-memory computation array and the second in-memory computation array are configured to process the first receptive field and the second receptive field in parallel.

[0010] A device includes first and second means for performing bitwise operations and bit counting on a set of activations. The first means is configured to store a plurality of weights corresponding to a filter tensor, receive a first set of activations corresponding to a first receptive field of an input, and process the first set of activations using the weights to generate a corresponding first tensor of output values. The second means is configured to store a first copy of the weights corresponding to the filter tensor, receive a second set of activations corresponding to a second receptive field of the input, and process the second set of activations using the first copy of the weights to generate a corresponding second tensor of output values. The first and second means are configured to process the first and second receptive fields in parallel.

[0011] This has broadly outlined the features and technical advantages of the present disclosure so that the detailed description below may be better understood. Additional features and advantages of the present disclosure will be described below. It will be appreciated by those skilled in the art that the present disclosure may be readily used as a basis for modifying or designing other structures for carrying out the same purposes as the present disclosure. It will also be appreciated by those skilled in the art that such equivalent constructions do not depart from the teachings of the present disclosure as set forth in the appended claims. The novel features which are believed to be characteristic of the present disclosure, both in terms of its organization and method of operation, together with further objects and advantages, will be better understood when the following description is considered in conjunction with the accompanying drawings. However, it is to be clearly understood that each of the figures is provided for illustration and description purposes only and is not intended as a definition of limitations of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The features, nature and advantages of the present disclosure will become more apparent when the detailed description set forth below is read in conjunction with the accompanying drawings, in which like reference numerals designate correspondingly throughout.

[0014] Figure 1An example implementation of designing a neural network using a system on a chip (SOC) including a general-purpose processor in accordance with certain aspects of the present disclosure is illustrated.

[0015] Figure 2A 、 2B 2C are diagrams illustrating neural networks according to aspects of the present disclosure.

[0016] Figure 2D is a diagram illustrating an exemplary deep convolutional network (DCN) according to aspects of the present disclosure.

[0017] Figure 3 is a diagram illustrating an exemplary deep convolutional network (DCN) according to aspects of the present disclosure.

[0018] Figure 4 An exemplary convolutional layer architecture with a compute-in-memory (CIM) array is illustrated.

[0019] Figure 5 A neural network architecture for compute-in-memory (CIM) parallel processing in convolutional layers is illustrated in accordance with aspects of the present disclosure.

[0020] Figure 6 The architecture of a compute-in-memory (CIM) array for an artificial neural network according to aspects of the present disclosure is illustrated.

[0021] Figure 7 Illustrated are bit cells of an in-memory compute array according to aspects of the present disclosure.

[0022] Figure 8 Shown is a timing diagram of various signals used for compute-in-memory (CIM) array operations.

[0023] Figure 9 A method for parallel processing of convolutional layers of a convolutional neural network according to aspects of the present disclosure is explained.

[0024] Detailed description

[0025] The detailed description set forth below in conjunction with the accompanying drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. This detailed description includes specific details to provide a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.

[0026] Based on this teaching, it will be appreciated by those skilled in the art that the scope of the present disclosure is intended to cover any aspect of the present disclosure, regardless of whether it is implemented independently or in combination with any other aspect of the present disclosure. For example, any number of aspects set forth can be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods that are practiced using other structures, functionality, or structure and functionality that are supplementary to or different from the various aspects of the present disclosure set forth. It should be understood that any aspect of the present disclosure disclosed can be implemented by one or more elements of the claims.

[0027] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0028] Although specific aspects are described herein, numerous variations and permutations of these aspects fall within the scope of this disclosure. While some benefits and advantages of preferred aspects are mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. On the contrary, various aspects of this disclosure are intended to be broadly applicable to different technologies, system configurations, networks, and protocols, some of which are illustrated as examples in the accompanying drawings and the following description of preferred aspects. The detailed description and drawings merely illustrate and do not limit the disclosure, the scope of which is defined by the appended claims and their equivalents.

[0029] Conventionally, convolutional layers in convolutional neural networks (CNNs) use small filters. However, CNNs sometimes have a very large number of input and output channels. Therefore, filters are reused multiple times in multiply-and-accumulate (MAC) operations to generate all outputs, which takes many cycles and is time-consuming.

[0030] In some digital implementations (e.g., 32-bit or less), filters of a convolutional neural network can be digitally cloned. Although digitally cloning the filters reduces the number of cycles in the convolutional neural network, digitally cloning the filters significantly increases power consumption. This is because the digital cloning process involves moving filter weights and parallel activations, which consumes a lot of energy in a short period of time.

[0031] Binary neural networks (BNNs) with binary activations and weights, operated according to aspects of the present disclosure, achieve sufficient accuracy for most artificial intelligence (AI) tasks. Computation-in-memory (CIM) is one of the best ways to implement BNNs as an analog implementation. Using BNNs allows replacing MAC operations with bitwise operations like XNOR and bit counting. Implementing BNNs in CIMs allows for power and time savings. That is, each memory cell is also a computation cell. One advantage of computing BNNs in-memory is energy / power reduction.

[0032] Various aspects of the present disclosure relate to cloning or replicating convolutional layer filters / weights and performing parallel multiplication and accumulation functions based on in-memory computation implementations. The convolutional layer performing parallel processing can be part of a binary neural network. In one aspect, a weight array corresponding to a filter implemented in an in-memory computation array is received and loaded into a plurality of bit cells of the in-memory computation array. The filter and its corresponding weights are replicated. A parallel convolutional layer multiplication and accumulation (MAC) function is then performed using the filter and the filter's replica(s). Outputs are generated simultaneously from the filter and each of the filter's replica(s).

[0033] In one aspect, performing a parallel convolutional layer multiply and accumulate (MAC) function using a filter and a copy of the filter includes receiving a first receptive field input as a portion of the total input to be processed by the convolutional layer. The method also includes receiving a second receptive field input as another portion of the total input to be processed by the convolutional layer. The first receptive field input and the second receptive field input are processed simultaneously using the filter and the copy of the filter, respectively. Parallel processing is analogous to in-memory computation arrays.

[0034] Aspects of the present disclosure reduce the cycle time of a convolutional layer of a neural network by at least four times (4X), while maintaining competitive energy / power consumption compared to alternative digital implementations. For example, aspects of the present disclosure can achieve energy savings of more than 2X over a digital implementation (e.g., a 1-bit logic implementation without parallel processing) or 7X over a 1-bit logic implementation with parallel processing.

[0035] Figure 1 An example implementation of a system on a chip (SOC) 100 according to certain aspects of the present disclosure is illustrated, which may include a central processing unit (CPU) 102 or a multi-core CPU configured for parallel processing of convolutional layers of a convolutional neural network. Variables (e.g., neural signals and synaptic weights), system parameters associated with the computing device (e.g., a neural network with weights), delays, frequency bin information, and task information may be stored in a memory block associated with a neural processing unit (NPU) 108, a memory block associated with the CPU 102, a memory block associated with a graphics processing unit (GPU) 104, a memory block associated with a digital signal processor (DSP) 106, a memory block 118, or may be distributed across multiple blocks. Instructions executed at the CPU 102 may be loaded from a program memory associated with the CPU 102 or may be loaded from the memory block 118.

[0036] The SOC 100 may also include additional processing blocks tailored for specific functions, such as a GPU 104, a DSP 106, a connectivity block 110 (which may include fifth generation (5G) connectivity, fourth generation long term evolution (4G LTE) connectivity, Wi-Fi connectivity, USB connectivity, Bluetooth connectivity, etc.), and a multimedia processor 112 that may, for example, detect and recognize gestures. In one implementation, the NPU is implemented in the CPU, DSP, and / or GPU. The SOC 100 may also include a sensor processor 114, an image signal processor (ISP) 116, and / or a navigation module 120 (which may include a global positioning system).

[0037] The SOC 100 may be based on the ARM instruction set. In one aspect of the present disclosure, the instructions loaded into the general purpose processor 102 may include code for receiving a weight array corresponding to a filter of a computation array in a memory array. The general purpose processor 102 may also include code for loading the weight array into a plurality of bit cells of the computation array in memory for the filter. The general purpose processor 102 may further include code for replicating the filter. In addition, the general purpose processor 102 may include code for performing a parallel convolutional layer multiplication and accumulation (MAC) function using the filter of the computation array in memory and a copy of the filter.

[0038] Deep learning architectures perform object recognition tasks by learning to represent inputs at successively higher levels of abstraction at each layer, thereby constructing useful feature representations of the input data. In this way, deep learning addresses a major bottleneck of conventional machine learning. Before the advent of deep learning, machine learning approaches to object recognition problems might rely heavily on human-engineered features, perhaps combined with shallow classifiers. A shallow classifier might be a two-class linear classifier, for example, where the weighted sum of the feature vector components is compared to a threshold to predict which class the input belongs to. Human-engineered features might be templates or kernels customized for a specific problem domain by engineers with domain expertise. In contrast, deep learning architectures learn to represent features similar to those that a human engineer might design, but they do so through training. Furthermore, deep networks can learn to represent and recognize new types of features that humans might not have considered.

[0039] Deep learning architectures can learn hierarchies of features. For example, if a first layer is presented with visual data, it can learn to recognize relatively simple features (such as edges) in the input stream. In another example, if a first layer is presented with auditory data, it can learn to recognize spectral power in specific frequencies. A second layer, taking the output of the first layer as input, can learn to recognize combinations of features, such as simple shapes for visual data or combinations of sounds for auditory data. For example, higher layers can learn to represent complex shapes in visual data or words in auditory data. Even higher layers can learn to recognize common visual objects or spoken phrases.

[0040] Deep learning architectures can perform particularly well when applied to problems with a natural hierarchical structure. For example, the classification of motor vehicles can benefit from first learning to recognize wheels, windshields, and other features. These features can then be combined in different ways at higher levels to identify cars, trucks, and airplanes.

[0041] Neural networks can be designed with a variety of connectivity patterns. In a feedforward network, information is passed from lower layers to higher layers, with each neuron in a given layer communicating to neurons in higher layers. As described above, hierarchical representations can be constructed in successive layers of a feedforward network. Neural networks can also have reflow or feedback (also known as top-down) connections. In a reflow connection, the output from a neuron in a given layer can be communicated to another neuron in the same layer. The reflow architecture can help identify patterns that span more than one chunk of input data delivered sequentially to the neural network. The connections from neurons in a given layer to neurons in lower layers are called feedback (or top-down) connections. A network with many feedback connections can be helpful when the recognition of high-level concepts can assist in discerning specific low-level features of the input.

[0042] The connections between layers of a neural network can be fully connected or partially connected. Figure 2A Illustrated is an example of a fully connected neural network 202. In the fully connected neural network 202, a neuron in a first layer may communicate its output to every neuron in a second layer, such that every neuron in the second layer will receive input from every neuron in the first layer. Figure 2BAn example of a locally connected neural network 204 is illustrated. In the locally connected neural network 204, neurons in a first layer may be connected to a limited number of neurons in a second layer. More generally, the locally connected layers of the locally connected neural network 204 may be configured such that each neuron in a layer will have the same or similar connectivity pattern, but their connection strengths may have different values ​​(e.g., 210, 212, 214, and 216). The connectivity pattern of the local connections may produce spatially distinct receptive fields in higher layers due to the nature of the higher layer neurons in a given region receiving inputs that are tuned through training to a limited portion of the total input to the network.

[0043] An example of a locally connected neural network is a convolutional neural network. Figure 2C An example of a convolutional neural network 206 is illustrated. The convolutional neural network 206 can be configured such that the connection strengths associated with the inputs to each neuron in the second layer are shared (e.g., 208). Convolutional neural networks may be well suited for problems where the spatial location of the inputs is meaningful.

[0044] One type of convolutional neural network is a deep convolutional network (DCN). Figure 2D A detailed example of a DCN 200 designed to recognize visual features of an image 226 input from an image capture device 230 (such as an onboard camera) is illustrated. The DCN 200 of the current example can be trained to identify traffic signs and the numbers provided on traffic signs. Of course, the DCN 200 can be trained for other tasks, such as identifying lane markings or identifying traffic lights.

[0045] DCN 200 can be trained using supervised learning. During training, an image (such as image 226 of a speed limit sign) can be presented to DCN 200, and a "forward pass" can then be calculated to produce output 222. DCN 200 can include a feature extraction section and a classification section. Upon receiving image 226, convolution layer 232 can apply a convolution kernel (not shown) to image 226 to generate a first set of feature maps 218. As an example, the convolution kernel of convolution layer 232 can be a 5x5 kernel that generates a 28x28 feature map. In this example, since four different feature maps are generated in first set of feature maps 218, four different convolution kernels are applied to image 226 at convolution layer 232. Convolution kernels can also be referred to as filters or convolution filters.

[0046] The first feature map set 218 may be subsampled by a maximum pooling layer (not shown) to generate a second feature map set 220. The maximum pooling layer reduces the size of the first feature map set 218. That is, the size of the second feature map set 220 (such as 14x14) is smaller than the size of the first feature map set 218 (such as 28x28). The reduced size provides similar information to subsequent layers while reducing memory consumption. The second feature map set 220 may be further convolved via one or more subsequent convolutional layers (not shown) to generate one or more subsequent feature map sets (not shown).

[0047] exist Figure 2D In the example of , second feature map set 220 is convolved to generate first feature vector 224. In addition, first feature vector 224 is further convolved to generate second feature vector 228. Each feature of second feature vector 228 may include a number corresponding to a possible feature of image 226 (such as "sign," "60," and "100"). A softmax function (not shown) may convert the numbers in second feature vector 228 into probabilities. Thus, output 222 of DCN 200 is the probability that image 226 includes one or more features.

[0048] In this example, the probabilities for "logo" and "60" in output 222 are higher than the probabilities for other features of output 222 (such as "30," "40," "50," "70," "80," "90," and "100"). Before training, output 222 generated by DCN 200 is likely incorrect. Thus, the error between output 222 and the target output can be calculated. The target output is the true value of image 226 (e.g., "logo" and "60"). The weights of DCN 200 can then be adjusted to more closely align output 222 of DCN 200 with the target output.

[0049] To adjust the weights, the learning algorithm can calculate a gradient vector for the weights. This gradient can indicate the amount by which the error will increase or decrease if the weights are adjusted. At the top layer, the gradient can directly correspond to the value of the weights connecting the activated neurons in the penultimate layer to the neurons in the output layer. In lower layers, the gradient can depend on the value of the weights and the calculated error gradients for higher layers. The weights can then be adjusted to reduce the error. This method of adjusting weights can be called "backward propagation" because it involves a "backward pass" through the neural network.

[0050] In practice, the error gradient of the weights may be calculated over a small number of examples so that the calculated gradient approximates the true error gradient. This approximation method may be referred to as stochastic gradient descent. Stochastic gradient descent may be repeated until the error rate achievable by the entire system has stopped decreasing or until the error rate has reached a target level. After learning, the DCN may be presented with a new image (e.g., image 226 of a speed limit sign) and a forward pass through the network may produce output 222, which may be considered an inference or prediction of the DCN.

[0051] Deep Belief Network (DBN) is a probabilistic model including multiple layers of hidden nodes. DBN can extract a hierarchical representation of the training data set. DBN can be obtained by stacking several layers of restricted Boltzmann machines (RBM). RBM is an artificial neural network that can learn probability distributions on an input set. Since RBM can learn probability distributions without information about which class each input should be classified into, RBM is often used in unsupervised learning. Using a hybrid unsupervised and supervised paradigm, the bottom RBM of the DBN can be trained in an unsupervised manner and can be used as a feature extractor, while the top RBM can be trained in a supervised manner (on the joint distribution of the input and target class from the previous layer) and can be used as a classifier.

[0052] A deep convolutional network (DCN) is a network of convolutional networks with additional pooling and normalization layers. DCNs have achieved state-of-the-art performance on many tasks. DCNs can be trained using supervised learning, where both the input and output targets are known for many examples and are used to modify the network's weights using gradient descent.

[0053] The DCN can be a feedforward network. In addition, as described above, the connections from neurons in the first layer of the DCN to the neuron groups in the next higher layer are shared across the neurons in the first layer. The feedforward and shared connections of the DCN can be exploited for fast processing. The computational burden of the DCN can be much smaller than, for example, the computational burden of a similarly sized neural network that includes recurrent or feedback connections.

[0054] The processing of each layer of the convolutional network can be thought of as a spatially invariant template or basis projection. If the input is first decomposed into multiple channels, such as the red, green, and blue channels of a color image, then the convolutional network trained on this input can be thought of as three-dimensional, with two spatial dimensions along the axes of the image and a third dimension that captures color information. The output of the convolutional connection can be thought of as forming a feature map in the subsequent layer, each element in which receives input from a range of neurons in the previous layer (e.g., feature map 218) and from each of the multiple channels. The values ​​in the feature map can be further processed with nonlinearities (such as rectified max(0,x)). The values ​​from adjacent neurons can be further pooled (which corresponds to downsampling) and can provide additional local invariance and dimensionality reduction. Normalization can also be applied by lateral inhibition between neurons in the feature map, which corresponds to whitening.

[0055] The performance of deep learning architectures improves as more labeled data points become available or as computing power increases. Modern deep neural networks are routinely trained using thousands of times more computing resources than were available to typical researchers just fifteen years ago. New architectures and training paradigms can further boost deep learning performance. Rectified linear units can alleviate the training problem known as vanishing gradients. New training techniques can reduce overfitting and thus enable larger models to achieve better generalization. Encapsulation techniques can abstract the data within a given receptive field and further improve overall performance.

[0056] Figure 3 3 is a block diagram illustrating a deep convolutional network 350. The deep convolutional network 350 may include multiple layers of different types based on connectivity and weight sharing. Figure 3 As shown in FIG, the deep convolutional network 350 includes convolution blocks 354A and 354B. Each of the convolution blocks 354A and 354B may be configured with a convolution layer (CONV) 356, a normalization layer (LNorm) 358, and a maximum pooling layer (MAX POOL) 360.

[0057] The convolution layer 356 may include one or more convolution filters that can be applied to the input data to generate a feature map. Although only two convolution blocks 354A and 354B are shown, the present disclosure is not limited thereto, and instead any number of convolution blocks 354A and 354B may be included in the deep convolutional network 350 according to design preferences. The normalization layer 358 may normalize the output of the convolution filter. For example, the normalization layer 358 may provide whitening or lateral suppression. The maximum pooling layer 360 may provide spatial downsampling aggregation to achieve local invariance and dimensionality reduction.

[0058] For example, the parallel filter banks of the deep convolutional network can be offloaded onto the CPU 102 or GPU 104 of the SOC 100 to achieve high performance and low power consumption. In alternative embodiments, the parallel filter banks can be offloaded onto the DSP 106 or ISP 116 of the SOC 100. In addition, the deep convolutional network 350 can access other processing blocks that may be present on the SOC 100, such as the sensor processor 114 and the navigation module 120 dedicated to sensors and navigation, respectively.

[0059] The deep convolutional network 350 may also include one or more fully connected layers 362 (FC1 and FC2). The deep convolutional network 350 may further include a logistic regression (LR) layer 364. Between each layer 356, 358, 360, 362, 364 of the deep convolutional network 350 are weights (not shown) to be updated. The output of each layer (e.g., 356, 358, 360, 362, 364) can be used as input to a subsequent layer (e.g., 356, 358, 360, 362, 364) in the deep convolutional network 350 to learn a hierarchical feature representation from the input data 352 (e.g., image, audio, video, sensor data, and / or other input data) supplied from the first convolutional block 354A. The output of the deep convolutional network 350 is a classification score 366 for the input data 352. The classification score 366 can be a set of probabilities, where each probability is a probability that the input data includes a feature from a feature set.

[0060] Figure 4 An exemplary convolutional layer architecture 400 is illustrated with an in-memory computation array 408. The convolutional layer architecture 400 is part of a convolutional neural network (e.g., a binary neural network or the convolutional neural network 200 of FIG. 2 ). Convolutional neural networks can be designed with multi-dimensional (tensor) data. For example, the input 402 of the convolutional layer architecture 400 has dimensions of 1 by 38 by 11 (1 (depth) x 38 (height) x 11 (width)). The first layer or convolutional layer 1 (illustrated by the convolutional layer architecture 400) has sixty-four (64) output channels and, accordingly, 64 filters, represented by the filter tensor 414.

[0061] Each of the 64 filters in filter tensor 414 (e.g., exemplary filter 412) has dimensions of one by five by two (1x5x2) (collectively, these filters or filter tensor 414 equal a 64x5x2 filter). Each 1x5x2 filter (e.g., filter 412) is convolved with input 402 to generate a 1x34x10 layer of output 404 (collectively, output 404 has dimensions 64x34x10). Each layer of output 404 specifies three thousand four hundred (3400) multiplication and accumulate (MAC) (or equivalent) operations for 340 5x2 receptive fields (e.g., receptive field input 406) and corresponding filters (e.g., filter 412). Each value of output 404 specifies 10 multiplications (of a 5x2 filter by a 5x2 receptive field). For example, each 5x2 filter is convolved with 5x2 activations (10 MACs) to generate an output value (e.g., a number) in output 404.

[0062] Six hundred and forty (640) weights of all filters 414 (64x5x2) may be stored in a compute in memory (CIM) array 408. Subsequently, ten (10) activations of each of the 340 5x2 receptive fields (e.g., receptive field input 406) are input to the CIM array 408 three hundred and forty (340) times using a word line (e.g., 416) and multiplied by the corresponding weights to produce a 1x1x64 output tensor (e.g., output tensor 410). The total output tensor 404 is the accumulation of the 1x1x64 output tensors of all receptive fields (e.g., receptive field input 406) of the input 402. For simplicity, Figure 4 The in-memory computing array 408 shows only some illustrative lines for inputs and outputs in the in-memory computing array 408.

[0063] The in-memory computation array 408 includes word lines 416 (e.g., word lines corresponding to rows of the in-memory computation array 408) through which the CIM array 408 receives the receptive fields (e.g., receptive field inputs 406) and bit lines 418 (corresponding to columns of the in-memory computation array 408). The in-memory computation array 408 also includes a global count word line (PCWL) and a read word line RWL ( Figure 4 Not explained in Figure 7416). Word lines 416 are used for initial weight definition. However, once the initial weight definition occurs, the activation input activates the specially designed lines corresponding to PCWL in the CIM bit cells to perform MAC operations. Each intersection of a bit line 418 and a word line 416 represents a filter weight value, which is multiplied by the input activation on word line 416 to generate a product. The volumes along each bit line 418 are then summed to generate the corresponding output value of the output tensor 410. The summed value can be charge, current, or voltage. In each operation cycle, ten values ​​corresponding to the receptive field are input on word line 416.

[0064] The multiplications performed in the in-memory computation array 408 are summed and output at bit lines 418 of the in-memory computation array 408. The output tensor 404 after processing the entire input 402 of the convolutional layer has dimensions of sixty-four by thirty-four by ten (64x34x10). However, the in-memory computation array 408 only generates 64 filter outputs at a time. Consequently, the multiplication and accumulation operations for the entire input 402 are delayed because only 10 values ​​in the corresponding receptive field of the entire input 402 are input to the word lines in each computation cycle, resulting in an increased number of cycles to process the entire input 402. For example, the processing of the entire input 402 can be completed in 34x10, or 340 cycles. Consequently, the filters are reused multiple times in the multiplication and accumulation (MAC) operations to generate all outputs 404, which takes many cycles and is time-consuming.

[0065] Figure 5 An exemplary neural network architecture 500 for in-memory computational parallel processing in convolutional layers according to aspects of the present disclosure is illustrated. For illustrative purposes, Figure 4 Some of the equipment and features of Figure 5 The convolution layer includes multiple (e.g., four) parallel in-memory computation arrays 508a, 508b, 508c, and 508d to simultaneously process multiple receptive field inputs of the entire input 502 in parallel. Each of the parallel in-memory computation arrays 508a, 508b, 508c, and 508d can be similar to Figure 4 The in-memory calculation array 408. The input 502 can be similar to Figure 4 Input 402.

[0066] In one aspect, the in-memory computation arrays 508a, 508b, 508c, and 508d are copies (or replicas) of each other. Each filter (e.g., exemplary filter 512) of each of the in-memory computation arrays 508a, 508b, 508c, and 508d has dimensions of one by five by two (1x5x2) (all together, the sixty-four (64) filters of the filter tensor 514 equal one 64x5x2 filter). By using each of the in-memory computation arrays 508a, 508b, 508c, and 508d, six hundred and forty (640) weights for all filters (64x5x2) are stored in each of the in-memory computation arrays 508a, 508b, 508c, and 508d. Thus, in-memory computation arrays 508 a , 508 b , 508 c , and 508 d include weight arrays corresponding to filters that are identical to the weight arrays corresponding to the filters of in-memory computation array 408 .

[0067] The plurality of receptive field inputs of the entire input 502 may include a first receptive field input 506a (e.g., similar to Figure 4 Receptive field input 406), second receptive field input 506b, third receptive field input 506c and fourth receptive field input 506d. Figure 4 Each of the receptive field inputs 506a, 506b, 506c, and 506d has a dimension of 1x5x2. Adjacent receptive field inputs of the receptive field inputs 506a, 506b, 506c, and 506d overlap and share some values. The multiple receptive field inputs 506a, 506b, 506c, and 506d can be processed simultaneously in multiple (e.g., four) parallel CIM arrays 508a, 508b, 508c, and 508d, respectively, to generate a Figure 4 The architecture 400 processes the input 502 within a quarter of the cycle required to process the input 402. The multiple parallel CIM arrays 508a, 508b, 508c, and 508d can simultaneously generate multiple receptive field outputs 510a, 510b, 510c, and 510d. Figure 4 Convolution using multiple parallel CIM arrays 508a, 508b, 508c, and 508d simultaneously takes only eighty-five (85) cycles, as opposed to the three hundred and forty (340) cycles of the architecture 400. Accordingly, the overall energy usage of the neural network architecture 500 can be reduced using parallel processing. Note that while an exemplary implementation using four CIM arrays has been described above, alternative implementations can have any suitable number of CIM arrays operating in parallel (with corresponding modifications).

[0068] Figure 6An exemplary architecture 600 of a compute-in-memory (CIM) array for an artificial neural network according to aspects of the present disclosure is illustrated. The CIM array can be used to implement the parallel processing techniques described above. Compute-in-memory is a technique for performing multiplication and accumulation operations within a memory array. The memory cells are also processing units. The CIM array includes population counter word lines (PCWLs) 625a, 625b, and 625c, read word lines (RWLs) 627a, 627b, and 627c, analog-to-digital converters (ADCs) 611a, 611b, and 611c, a digital processing unit 613, and bit lines 618a, 618b, and 618c. Weights (e.g., binary synaptic weight matrices) are stored in SRAM cells. Exemplary binary weights are shown in the SRAM cells for illustration. Figure 6 Nine bit cells 605a-605i of the in-memory computation array are shown. Input activations (which may be, for example, input values ​​of an input vector) are provided on PCWL 625.

[0069] Multiplication occurs in each bit cell associated with a bit line, and the accumulation (summing) of the results of all bit cell multiplications occurs on the same bit line for a column. The multiplication in each bit cell is in the form of an AND operation of the corresponding activation and weight, where the product is stored as a charge on the corresponding capacitor 623. For example, only when the activation is 1 (here, because PMOS is used, PCWL is 0 for activation 1) and the weight is 1, a product of 1 is generated, and therefore a charge on capacitor 623 is generated.

[0070] For example, according to this aspect, bit line 618a accumulates the charges of all bit cells in the first column. The accumulated or summed charges include the summed charge 603 of bit line 618a, the summed charge 607 of bit line 618b, and the summed charge 609 of bit line 618c. The summed charge 603 of bit line 618a is passed to a comparator or ADC 611a, where it is converted into a digital output (e.g., a digital signal / number), which is the input to the next layer of the neural network. The digital outputs in each bit line may or may not be summed together, depending on whether it is a single-bit or multi-bit neural network (NN) operation. When the activations on the total count word lines (PCWL) 625a, 625b, and 625c are, for example, 1, 0, and 1, the sums of bit lines 618a-c are 0+0+1=1, 1+0+0=1, and 1+0+1=2, respectively. The outputs of ADCs 611a, 611b, and 611c are passed to digital processing unit 613 for further processing.

[0071] Figure 7 Illustrated is an exemplary bit cell 700 of an in-memory computation array according to aspects of the present disclosure (e.g., Figure 66T) is coupled to ground.

[0072] Programming the weights in the bit cells can be performed once for many activations. For example, in operation, the SRAM cell 701 receives only one bit of information at nodes 717 and 719 via word line 716. For example, when the information received at node 717 is "1" (e.g., a high voltage), the other node 719 does not receive information, or the information received at node 719 is "0" (e.g., a low voltage). Similarly, when the information received at node 717 is "0", the information received at node 719 is "1". Weight programming can be followed by activation input and multiplication steps to charge the capacitor according to the corresponding product. Transistor 711 is activated by an activation signal (PCWL signal) through the overall count word line (PCWL) 725 of the in-memory computing array, and transistor 713 is activated by a signal (RWL signal) through another word line (e.g., read word line (RWL) 727) of the in-memory computing array. Read word line 727 is provided to all bit cells in a row and is turned on to accumulate the values ​​of the bit cells.

[0073] If node 717 is "0" (e.g., when the stored weight value is "0"), capacitor 723 will not be charged even if low PCWL indicates an activation "1" at the gate of transistor 711. Accordingly, no charge is provided to bit line 721. However, if node 717, which corresponds to the weight value, is "1" and PCWL is set low (e.g., when the activation input is high), this turns on PMOS transistor 711, which acts as a short circuit, thereby allowing capacitor 723 to be charged. After capacitor 723 is charged, transistor 711 is turned off, so that charge is stored in capacitor 723. To move charge from capacitor 723 to bit line 721, NMOS transistor 713 is turned on by RWL 727, causing NMOS transistor 713 to act as a short circuit.

[0074] Figure 8 A timing diagram 800 illustrating the timing of various signals during a compute-in-memory (CIM) array operation is shown. The first row of the timing diagram 800 shows the global count wordline PCWL going low (e.g., Figure 6 625a or Figure 7 725). A low PCWL indicates an active "1". When PCWL is low, the PMOS transistor turns on, which allows the capacitor to charge (if the weight is "1"). The second row shows the read word line RWL (e.g., Figure 6 The read word line 627a or Figure 7 727). The third row shows the read bit line RBL (e.g., 618 or 721), the fourth row shows the analog-to-digital converter (ADC) readout signal, and the fifth row shows the reset signal. For example, referring to Figure 7 When the read word line RWL is high, the charge from the capacitor 723 is gradually transferred to the read bit line RBL.

[0075] Sum the charge / current / voltage (e.g. Figure 6 603 or from Figure 7 The summed charge of the bit line 721 is passed to a comparator or ADC (e.g., Figure 6 The summed charge is converted to a digital output (e.g., a digital signal / number) by the ADC 611a. The summation of the charge may occur in the accumulation region of the timing diagram 800, and the readout from the ADC may be associated with the ADC readout region of the timing diagram 800. After the ADC readout is obtained, a reset signal discharges all capacitors (e.g., capacitor 623) in preparation for processing the next set of active inputs.

[0076]

[0077]

[0078] Table 1

[0079] Table 1 illustrates an example of an in-memory compute array operation. The first column of Table 1 includes possible values ​​for the incoming activation signal. According to aspects of the present disclosure, the second column includes the PCWL that activates transistors designed to implement the in-memory compute array. Because transistor 711 is a PMOS transistor, the PCWL value is the inverse of the activation value. For example, the in-memory compute array includes transistor 711 that is activated by an activation signal (PCWL signal) via a global count wordline (PCWL) 725.

[0080] The third column includes a plurality of weights corresponding to the filter tensor stored in the SRAM cell nodes of the in-memory computation array. The fourth column shows the resulting product that will be stored as charge on the capacitor. For example, the charge may be stored at the node of capacitor 723 or at the node of one of capacitors 623a-623i. When transistor 713 is activated, the charge from capacitor 723 is moved to bit line 721. For example, referring to transistor 711, when the weight at cell node 717 is "1" (e.g., a high voltage) and the input activation is "1" (so PCWL is "0"), capacitor 723 is charged (e.g., the node of the capacitor is "1"). For all other combinations, the capacitor node will have a value of 0.

[0081] The parallel processing techniques disclosed herein can facilitate any type of edge computing involving artificial neural networks. These techniques have applicability during the inference phase or any other phase of neural network processing. Binary networks are suitable when high accuracy is not required. While the discussion primarily focuses on binary neural networks, other low-bitwidth neural networks are also contemplated.

[0082] Figure 9 A method 900 for parallel processing of convolutional layers of a convolutional neural network according to aspects of the present disclosure is illustrated. Figure 9 As shown, at box 902, a plurality of weights corresponding to a filter tensor are stored in a first in-memory computation array. At box 904, a first set of activations corresponding to a first receptive field of an input is received by the first in-memory computation array. At box 906, a first copy of the weights corresponding to the filter tensor is stored in a second in-memory computation array. At box 908, a second set of activations corresponding to a second receptive field of the input is received by the second in-memory computation array. At box 910, the first set of activations is processed in the first in-memory computation array using the weights to generate a corresponding first tensor of output values, and the second set of activations is processed in the second in-memory computation array using the first copy of the weights to generate a corresponding second tensor of output values. The first in-memory computation array and the second in-memory computation array are configured to process the first receptive field and the second receptive field in parallel.

[0083] According to a further aspect of the present disclosure, a parallel processing device for a convolutional layer of a convolutional neural network is described. The parallel processing device includes a first device for performing bitwise operations and bit counts on a set of activations. The first device includes an in-memory computation array 408 or 508a, a deep convolutional network 200, a deep convolutional network 350, a convolutional layer 232, a SoC 100, a CPU 102, and / or a convolution block 354A. The parallel processing device further includes a second device for performing bitwise operations and bit counts on the set of activations. The second device includes an in-memory computation array 508b, a deep convolutional network 200, a deep convolutional network 350, a convolutional layer 232, a SoC 100, a CPU 102, and / or a convolution block 354A.

[0084] The parallel processing apparatus further includes a third means for performing bitwise operations and bit counting on the set of activations. The third means includes the in-memory computation array 508c, the deep convolutional network 200, the deep convolutional network 350, the convolutional layer 232, the SoC 100, the CPU 102, and / or the convolution block 354A. In another aspect, the aforementioned means may be any module or any device configured to perform the functions recited by the aforementioned means.

[0085] The various operations of the methods described above may be performed by any suitable device capable of performing the corresponding functions. These devices may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs), or processors. Generally speaking, where there are operations illustrated in the figures, those operations may have corresponding counterpart means-plus-function components with similar numbering.

[0086] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, database, or another data structure), ascertaining, and the like. Additionally, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Furthermore, "determining" may include resolving, selecting, choosing, establishing, and the like.

[0087] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. As an example, "at least one of a, b, or c" is intended to cover: a, b, c, ab, ac, bc, and abc.

[0088] The various illustrative logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0089] The steps of the method or algorithm described in conjunction with the present disclosure can be implemented directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module can reside in any form of storage medium known in the art. Some examples of usable storage media include random access memory (RAM), read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs, and the like. The software module can include a single instruction or many instructions and can be distributed over several different code segments, distributed between different programs, and distributed across multiple storage media. A storage medium can be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative, a storage medium can be integrated into a processor.

[0090] The methods disclosed herein include one or more steps or actions for implementing the described methods. These method steps and / or actions may be interchangeable with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of the specific steps and / or actions may be modified without departing from the scope of the claims.

[0091] The functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in hardware, an example hardware configuration may include a processing system in a device. The processing system may be implemented using a bus architecture. Depending on the specific application and overall design constraints of the processing system, the bus may include any number of interconnecting buses and bridges. The bus may link together various circuits including a processor, a machine-readable medium, and a bus interface. The bus interface may connect a network adapter, etc., to the processing system via the bus. The network adapter may implement signal processing functions. For certain aspects, a user interface (e.g., a keypad, a display, a mouse, a joystick, etc.) may also be connected to the bus. The bus may also link various other circuits, such as timing sources, peripherals, voltage regulators, power management circuits, and the like, which are well known in the art and will not be described further.

[0092] The processor may be responsible for managing the bus and general processing, including the execution of software stored on the machine-readable medium. The processor may be implemented with one or more general and / or special processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuit systems that can execute software. Software should be broadly interpreted to mean instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or other. As an example, the machine-readable medium may include random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. The machine-readable medium may be implemented in a computer program product. The computer program product may include packaging material.

[0093] In a hardware implementation, the machine-readable medium can be a part of the processing system that is separate from the processor. However, as will be readily appreciated by those skilled in the art, the machine-readable medium or any part thereof can be external to the processing system. As an example, the machine-readable medium can include a transmission line, a carrier wave modulated by data, and / or a computer product separate from the device, all of which can be accessed by the processor via a bus interface. Alternatively or in addition, the machine-readable medium or any part thereof can be integrated into the processor, as may be the case with a cache and / or general register file. Although the various components discussed can be described as having specific locations, such as local components, they can also be configured in various ways, such as with certain components being configured as part of a distributed computing system.

[0094] The processing system can be configured as a general-purpose processing system having one or more microprocessors providing processor functionality and external memory providing at least a portion of the machine-readable medium, all linked together with other supporting circuitry via an external bus architecture. Alternatively, the processing system can include one or more neuromorphic processors for implementing the neuron model and neural system model described herein. As another alternative, the processing system can be implemented using an application-specific integrated circuit (ASIC) with a processor, bus interface, user interface, supporting circuitry, and at least a portion of the machine-readable medium integrated into a single chip, or using one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gating logic, discrete hardware components, or any other suitable circuitry, or any combination of circuits capable of performing the various functionalities described throughout this disclosure. Depending on the specific application and the overall design constraints imposed on the overall system, one skilled in the art will recognize how to best implement the functionality described with respect to the processing system.

[0095] The machine-readable medium may include several software modules. These software modules include instructions that, when executed by a processor, cause a processing system to perform various functions. These software modules may include a transmitting module and a receiving module. Each software module may reside in a single storage device or be distributed across multiple storage devices. As an example, when a triggering event occurs, a software module may be loaded from a hard drive into RAM. During execution of the software module, the processor may load some instructions into a cache to increase access speed. One or more cache lines may then be loaded into a general register file for execution by the processor. When describing the functionality of a software module below, it will be understood that such functionality is implemented by the processor when the processor executes instructions from the software module. In addition, it should be appreciated that various aspects of the present disclosure result in improvements to the functionality of a processor, computer, machine, or other system that implements such aspects.

[0096] If implemented in software, each function may be stored as one or more instructions or codes on or transmitted by a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared (IR), radio, and microwave), then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (such as infrared, radio, and microwave) is included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Thus, in some aspects, computer-readable media may include non-transitory computer-readable media (e.g., tangible media). Additionally, for other aspects, computer-readable media may include transient computer-readable media (e.g., signals). Combinations of the above should also be included within the scope of computer-readable media.

[0097] Thus, some aspects may include a computer program product for performing the operations presented herein. For example, such a computer program product may include a computer-readable medium having stored (and / or encoded) thereon instructions, which are executable by one or more processors to perform the operations described herein. For some aspects, the computer program product may include packaging materials.

[0098] In addition, it should be appreciated that the modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by the user terminal and / or base station where applicable. For example, such a device can be coupled to a server to facilitate the transfer of the means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage device (e.g., RAM, ROM, a physical storage medium such as a compact disc (CD) or floppy disk, etc.) so that once the storage device is coupled to or provided to the user terminal and / or base station, the device can obtain the various methods. In addition, any other suitable technology suitable for providing the methods and techniques described herein to a device can be utilized.

[0099] It will be understood that the claims are not limited to the precise configuration and components illustrated above. Various changes, substitutions and variations may be made in the arrangement, operation and details of the methods and apparatus described above without departing from the scope of the claims.

Claims

1. A device comprising: A first in-memory computation array configured to: Stores multiple weights corresponding to the filter tensor; receiving a first set of activations corresponding to a first receptive field of the input; as well as processing the first set of activations using the plurality of weights to generate a corresponding first tensor of output values; as well as A second in-memory computation array is configured to: storing a first copy of the plurality of weights corresponding to the filter tensor; receiving a second set of activations corresponding to a second receptive field of the input; as well as The second set of activations is processed using the first copy of the plurality of weights to generate a corresponding second tensor of output values, wherein the first in-memory computation array and the second in-memory computation array are configured to process the first receptive field and the second receptive field in parallel.

2. The apparatus of claim 1 , wherein the first in-memory computation array and the second in-memory computation array are configured to simultaneously perform a parallel convolutional layer multiplication and accumulation (MAC) function on the first receptive field and the second receptive field.

3. The apparatus of claim 2, wherein the parallel convolutional layer multiplication and accumulation functionality comprises analog processes in the first in-memory computation array and the second in-memory computation array.

4. The apparatus of claim 1 , wherein the apparatus is used for a convolutional neural network, and the convolutional neural network comprises a binary neural network.

5. The apparatus of claim 1 , wherein the filter tensor comprises a plurality of filters corresponding to output channels of a convolutional neural network layer.

6. The apparatus of claim 1 , further comprising: A third in-memory computation array is configured to: storing a second copy of the plurality of weights corresponding to the filter tensor; receiving a third set of activations corresponding to a third receptive field of the input; as well as The third set of activations is processed using the second copy of the plurality of weights to generate a corresponding third tensor of output values, the first in-memory computation array, the second in-memory computation array, and the third in-memory computation array being configured to process the first, second, and third receptive fields in parallel.

7. The apparatus of claim 1, wherein processing the first set of activations and the second set of activations comprises bitwise operations and bit counting.

8. A method for processing a convolutional layer of a convolutional neural network, comprising: storing a plurality of weights corresponding to the filter tensor in a first in-memory computation array; Receiving, by the first in-memory computation array, a first set of activations corresponding to a first receptive field of an input; storing a first copy of the plurality of weights corresponding to the filter tensor in a second in-memory computation array; receiving, by the second in-memory computation array, a second set of activations corresponding to a second receptive field of the input; as well as The first set of activations is processed in the first in-memory computation array using the plurality of weights to generate a corresponding first tensor of output values, and the second set of activations is processed in the second in-memory computation array using the first copy of the plurality of weights to generate a corresponding second tensor of output values, wherein the first in-memory computation array and the second in-memory computation array are configured to process the first receptive field and the second receptive field in parallel.

9. The method of claim 8, wherein the filter tensor comprises a plurality of filters corresponding to output channels of a convolutional neural network layer.

10. The method of claim 8, wherein processing the first set of activations and the second set of activations comprises bitwise operations and bit counting.

11. The method of claim 8, further comprising: Parallel convolutional layer multiplication and accumulation (MAC) functions on the first receptive field and the second receptive field are performed in the first in-memory computation array and the second in-memory computation array, respectively.

12. The method of claim 11, wherein the parallel convolutional layer multiply and accumulate functionality comprises analog processes in the first in-memory computation array and the second in-memory computation array.

13. The method of claim 8, further comprising: storing a second copy of the plurality of weights corresponding to the filter tensor in a third in-memory computation array; receiving a third set of activations corresponding to a third receptive field of the input; as well as The third set of activations is processed using the second copy of the plurality of weights to generate a corresponding third tensor of output values, the first in-memory computation array, the second in-memory computation array, and the third in-memory computation array being configured to process the first, second, and third receptive fields in parallel.

14. The method of claim 8, wherein the convolutional neural network comprises a binary neural network.

15. A device comprising: first means for performing bitwise operations and bit counting on a set of activations, the first means being configured to: Stores multiple weights corresponding to the filter tensor; receiving a first set of activations corresponding to a first receptive field of the input; as well as processing the first set of activations using the plurality of weights to generate a corresponding first tensor of output values; as well as second means for performing bitwise operations and bit counting on the set of activations, the second means being configured to: storing a first copy of the plurality of weights corresponding to the filter tensor; receiving a second set of activations corresponding to a second receptive field of the input; as well as The second set of activations is processed using the first copy of the plurality of weights to generate a corresponding second tensor of output values, wherein the first device and the second device are configured to process the first receptive field and the second receptive field in parallel.

16. The apparatus of claim 15, wherein the first means and the second means are configured to simultaneously perform a parallel convolutional layer multiply and accumulate (MAC) function on the first receptive field and the second receptive field.

17. The apparatus of claim 16, wherein the parallel convolutional layer multiply and accumulate functionality comprises analog processes in the first device and the second device.

18. The apparatus of claim 15, wherein the apparatus is for a convolutional neural network, and the convolutional neural network comprises a binary neural network.

19. The apparatus of claim 15, wherein the filter tensor comprises a plurality of filters corresponding to output channels of a convolutional neural network layer.

20. The apparatus of claim 15, further comprising: third means for performing bitwise operations and bit counting on the set of activations, the third means being configured to: storing a second copy of the plurality of weights corresponding to the filter tensor; receiving a third set of activations corresponding to a third receptive field of the input; as well as The third set of activations is processed using the second copy of the plurality of weights to generate a corresponding third tensor of output values, wherein the first device, the second device, and the third device are configured to process the first, second, and third receptive fields in parallel.

Images