Depth-wise Separable Convolution on Multi-dimensional Memory Fabric
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in efficiently processing large images with 1 billion pixels due to data loading bottlenecks, leading to poor computing performance when running depth-wise convolution operations on neural networks, especially on two-dimensional arrays of compute cores.
Innovation Solution
The proposed method involves partitioning the input image into subimages and distributing them across a two-dimensional array of compute cores, preloading data to ensure it remains resident on the cores during the convolution operation, and using a data filter to manage data distribution efficiently, allowing for parallel arithmetic and data exchange operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is loaded from external storage during convolution operation, then data availability is maintained, but compute cores experience waiting time and reduced productivity
Solution Approach 1:
The patent pre-loads image data into on-chip memory of compute cores before the convolution operation begins. This preliminary action ensures that all necessary pixel data is already available in fast on-chip memory during the computation phase, eliminating waiting time and keeping compute cores continuously productive throughout the convolution operation.
Solution Approach 2:
The patent divides the large image into multiple tiles or blocks, with each compute core responsible for processing a specific tile. This segmentation allows parallel processing across multiple cores while each core operates independently on its assigned data portion, maximizing utilization without requiring continuous external data loading.
2Speed
If the entire image is loaded into on-chip memory, then data access speed improves, but memory capacity requirements increase
Solution Approach 1:
The patent segments the large image into smaller tiles that can be processed independently by different compute cores. Each compute core only needs to store and process its assigned tile in on-chip memory, dramatically reducing the memory capacity requirement per core while maintaining fast access speeds for the relevant data portion.
Solution Approach 2:
The patent introduces a spatial dimension by distributing image tiles across multiple compute cores in a two-dimensional array architecture. This transforms the single-core memory capacity problem into a distributed multi-core memory system, where total memory capacity is shared across cores, effectively reducing the memory burden on individual cores while maintaining overall system capacity.
3Quantity of substance
If convolution operation is performed sequentially, then memory requirements are reduced, but processing time increases
Solution Approach 1:
The patent segments both the image data and the convolution computation across multiple compute cores. Each core performs convolution on its assigned tile independently and in parallel with other cores, maintaining low per-core memory usage while achieving significant speedup through parallel execution of the segmented computational tasks.
Solution Approach 2:
The patent combines multiple parallel convolution operations across compute cores into a single unified processing stage. By merging the computational efforts of multiple cores working in parallel on different image tiles, the system achieves fast processing while each core maintains efficient memory utilization.
Data Source
AI summary
Embodiments of the present disclosure are directed to methods and systems for computing depth-wise convolutions on a two-dimensional array (or grid) of compute cores. In a first aspect of the disclosure, a method for running the depth-wise convolution on a two-dimensional array of compute cores comprises of (a) partitioning the input image among the compute cores, such that the compute core array collectively stores the whole image (b) allocating a memory buffer (an accumulator) that holds the subimage plus some frame (or padding) around the subimage (c) receiving convolution filter weights, multiplying the input subimage by the weight and adding to the accumulator with an offset (d) exchanging the information from the subimage frame with the neighboring compute cores. The advantage of the method is that the depth-wise convolution operation can be parallelized across a two-dimensional array of compute cores. In other words, this method enables acceleration of the depth-wise convolution operation (and therefore acceleration of the neural network operation). The depth-wise convolution method comprises (a) receiving the weights of the convolution filter (b) accumulator aliasing, i.e. writing into the same accumulator memory with an offset (c) exchanging small amounts of information with the neighbors.