Kernel Decomposition for NPU Convolution Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural processing units (NPUs) are often optimized for specific kernel sizes, such as 3×3, limiting their flexibility and efficiency in handling larger kernels, which are needed to capture more expressive features but incur higher computational costs.
Innovation Solution
The method decomposes larger kernels into sub-kernels with dimensions less than or equal to a target kernel dimension, allowing for efficient convolution operations by performing convolutions with each sub-kernel and summing the results to generate the output data array, thereby optimizing hardware capabilities for a range of kernel sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If larger kernels are used to capture more expressive features, then feature extraction capability is improved, but computational cost increases
Solution Approach 1:
The patent divides a large kernel into multiple smaller sub-kernels, each of which can be processed independently by the NPU's MAC units. This segmentation allows the system to maintain the expressive power of large kernels while breaking down the computational workload into manageable smaller operations that fit within the hardware constraints.
2Productivity
If NPUs are optimized for specific kernel sizes, then processing efficiency for those kernels is improved, but flexibility to handle various kernel sizes deteriorates
Solution Approach 1:
The patent enables the NPU to handle multiple kernel sizes by decomposing any kernel into sub-kernels that can be processed by the existing MAC unit architecture. This makes the NPU universally applicable to different kernel dimensions without requiring separate hardware optimizations for each kernel size.
Data Source
AI summary
A computer-implemented method of performing a convolution between an input data array and a kernel to generate an output data array includes decomposing the kernel into a plurality of sub-kernels each having a respective position relative to the kernel and respective in-plane dimensions less than or equal to a target kernel dimension, and for each of the plurality sub-kernels: determining a respective portion of the input data array on the basis of the respective in-plane dimensions of the sub-kernel and the respective position of the sub-kernel relative to the kernel; retrieving the respective portion of the input data array; and performing a convolution between the retrieved respective portion of the input data array and the sub-kernel to generate a respective intermediate data array. The method further includes summing the generated intermediate data arrays to generate at least a portion of the output data array.