Neural Network Accelerator Tile Grouping for Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network accelerators face inefficiencies in convolution operations due to high execution times, particularly in deep neural networks, as they do not effectively utilize data similarity to optimize memory access and parallel processing.
Innovation Solution
The proposed solution involves a scheduling method that identifies data tiles with high similarity, groups them into sets, and allocates these sets to components for parallel General Matrix Multiplication (GEMM) operations, reducing reliance on cache memory size and increasing main memory usage, thereby enhancing parallel processing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data tiles are processed sequentially without grouping by similarity, then the processing logic is simple, but the execution time is long and cache utilization is poor
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the similarity matrix between data tiles before processing. This allows the scheduler to quickly identify and group similar data tiles without performing complex similarity calculations during execution, thereby improving convolution operation speed while keeping the scheduling logic manageable.
Solution Approach 2:
The patent segments the processing workload by dividing data tiles into groups based on their similarity metrics. Each group of similar data tiles is processed together by different processing units, which improves cache utilization and parallel processing efficiency while maintaining organized and manageable scheduling complexity.
2Speed
If cache memory size is increased to store more data tiles, then data access speed improves, but hardware cost and complexity increase
Solution Approach 1:
The patent changes the parameter of data tile organization by sorting and grouping data tiles according to their similarity metrics before processing. This transformation allows the system to achieve high cache utilization with standard cache sizes, improving data access speed without requiring increased cache memory capacity or more complex memory architecture.
3Productivity
If data tiles with high similarity are grouped and processed in parallel, then cache utilization improves, but the scheduling algorithm becomes more complex
Solution Approach 1:
The patent applies preliminary action by pre-computing the similarity matrix between all data tiles and storing it in memory before the actual processing begins. This preliminary calculation enables the scheduling algorithm to simply look up and group similar data tiles based on pre-established similarity relationships, achieving high parallel processing efficiency while keeping the scheduling algorithm itself relatively simple.
Data Source
AI summary
The disclosure includes a memory storing at least one instruction, and at least one processor configured to execute the at least one instruction stored in the memory, wherein the at least one processor executes the at least one instruction to identify a first array of a plurality of data tiles constituting an unfolded input tensor obtained by unfolding an input tensor to perform a convolution operation by using a general matrix multiplication (GEMM) operation, identify a tile distance indicating a distance between a pair of data tiles with highest data similarity among the plurality of data tiles in the first array, form a plurality of data tile sets by grouping the plurality of data tiles based on the tile distance, and allocate the plurality of data tile sets to a plurality of components that process the general matrix multiplication operation in parallel.