Winograd Kernel Transformation in Neural Processors for CPU Offload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks rely heavily on central processing units (CPUs) for convolution operations, leading to significant CPU bandwidth consumption and increased power consumption.
Innovation Solution
A neural processor circuit with a data storage device and neural engine circuit that performs convolutions using a Winograd transform, incorporating multipliers, accumulators, and a kernel transformation circuit to generate intermediate kernel parameters, allowing for parallel processing of convolutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If convolutions are performed using CPU, then computational flexibility is maintained, but CPU bandwidth consumption increases and power consumption increases
Solution Approach 1:
The system segments convolution computations into two parts: kernel transformation (performed once per kernel) and convolution execution (performed per input frame). The pre-computed kernel parameters are stored in a buffer, separating the computationally intensive transformation from the repeated convolution operations, thereby reducing CPU bandwidth and power consumption during frame processing.
Solution Approach 2:
The kernel transformation is performed in advance before the convolution operations. The neural engine pre-computes kernel parameters from the original kernel data and stores them in a buffer, so that during actual convolution execution, only multiplication and accumulation operations are needed, significantly reducing real-time computational load and power consumption.
2Adaptability or versatility
If convolutions are performed using CPU, then general-purpose computing is maintained, but processing speed decreases
Solution Approach 1:
The neural engine acts as an intermediary between the CPU and the convolution operation. It receives kernel data from the CPU, performs the computationally intensive kernel transformation, stores pre-computed kernel parameters in a buffer, and then executes convolutions using only simple multiplication and accumulation operations, thereby accelerating processing speed while maintaining flexibility.
Solution Approach 2:
The system replaces the general-purpose CPU's mechanical computation process with a specialized neural engine that uses a dedicated pipeline architecture. The neural engine substitutes complex sequential CPU instructions with parallel multiplication and accumulation units, achieving faster processing speed for convolution operations.
3Measurement precision
If kernel transformation is performed for each convolution, then computation accuracy is maintained, but computational complexity increases
Solution Approach 1:
The system merges multiple convolution operations that use the same kernel into a single kernel transformation process. The pre-computed kernel parameters are stored in a buffer and reused across multiple convolutions, reducing computational complexity while maintaining accuracy through the mathematically equivalent Winograd transform.
Solution Approach 2:
The pre-computed kernel parameters serve multiple convolution operations universally. A single kernel transformation produces parameters that can be used for multiple different input frames and convolution operations, reducing overall computational complexity while maintaining computation accuracy through the universal applicability of the transformed parameters.
Data Source
AI summary
Embodiments relate to a neural processor circuit including an input transformation circuit and a neural engine circuit including a kernel transformation circuit configured to generate a number of intermediate kernel parameters. The input transformation circuit can generate, at a first time instance, a first set of intermediate input parameters corresponding to a first subsequence of input parameters for a first pair of convolutions based on the number of intermediate kernel parameters; generate, at the first time instance, a second set of intermediate input parameters corresponding to a second subsequence of input parameters for a second pair of convolutions based on the number of intermediate kernel parameters; and generate, at a second time instance after the first time instance, a third set of intermediate input parameters corresponding to a third subsequence of input parameters for a third pair of convolutions based on the number of intermediate kernel parameters.