RAPA Convolution for Analog Array Neural Network Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (ConvNets) do not map favorably onto analog arrays, as the convolution operation is formulated as a matrix-matrix product, leading to inefficient computation time due to the need for iterated matrix-vector products proportional to the total number of image pixels, and existing speedup strategies like data parallelism are not practical for analog arrays with stationary weights.
Innovation Solution
The Replicated Arrays with Permuted Assignment (RAPA) Convolution method replicates the kernel matrix onto multiple analog array tiles and distributes the computation equally among them, allowing for parallel processing of image patches and reducing overall computation time by using model parallelism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If convolution operation is formulated as matrix-matrix product on analog arrays, then computation can be performed using standard linear algebra operations, but computation time increases proportionally to the total number of image pixels due to iterated matrix-vector products
Solution Approach 1:
The patent divides the image into multiple patches and distributes them across multiple analog array tiles. Each tile processes a subset of image patches in parallel, transforming the single large matrix-matrix product into multiple smaller parallel operations. This segmentation reduces the computational burden on each individual analog array while maintaining the overall convolution function.
Solution Approach 2:
The patent introduces a new dimension of parallelism by utilizing multiple analog array tiles simultaneously. Instead of processing the entire image through a single sequential matrix-matrix product, the system distributes computation across multiple spatial dimensions (multiple tiles processing multiple patches), effectively adding a parallel processing dimension to the computation.
2Loss of time
If data parallelism is used to speed up computation, then computation time may be reduced, but it is not practical for analog arrays with stationary weights
Solution Approach 1:
The patent implements a dynamic mapping strategy where the assignment of image patches to analog array tiles is not fixed but can be permuted and reconfigured. This dynamic approach allows the system to adapt the distribution of computational tasks across tiles based on the specific convolution operation being performed, making the system practical for various ConvNet architectures while maintaining parallel processing benefits.
Solution Approach 2:
The patent segments both the image data and the analog array resources into manageable units (patches and tiles respectively). This segmentation enables a practical mapping strategy where each tile handles a specific subset of patches, making the parallel processing approach feasible and efficient for analog arrays with stationary weights.
3Loss of time
If kernel matrix is replicated onto multiple analog array tiles, then parallel processing of image patches is enabled and computation time is reduced, but device complexity increases
Solution Approach 1:
The patent replicates the kernel matrix across multiple analog array tiles, creating identical copies of the same computational logic. This copying approach enables parallel processing of different image patches without requiring complex inter-tile communication or coordination, as each tile independently performs the same convolution operation on its assigned patches using its local copy of the kernel matrix.
Solution Approach 2:
The replicated kernel matrices across multiple tiles provide universal computational capability, where each tile can process any image patch using the same set of filters. This multi-functionality allows the system to handle various ConvNet architectures and convolution operations without requiring specialized hardware configurations for each case.
Data Source
AI summary
Mechanisms are provided for acceleration of convolutional neural networks on analog arrays. Input ports receive image signals from frames in an input image. Input memory arrays store the image signals received from the input ports into a respective input memory location to create a plurality of image sub-regions in input memory arrays. A distributor associated each of a set of analog array tiles in an analog array to a part of image sub-regions of the input memory arrays, so that one or more of a set of analog memory components is associated with the image signals in a distribution order to create a respective output signal. An assembler stores each of the respective output signals into one of a set of memory outputs in an output order that is determined by the distribution order.