Winograd Neural Processor Circuit for Parallel Convolution Offload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks rely heavily on central processing units (CPUs) for convolutions, leading to significant CPU bandwidth consumption and increased power consumption.
Innovation Solution
A neural processor circuit with a data storage device and neural engine circuit that performs convolutions using a Winograd transform, incorporating multipliers, accumulators, and a kernel transformation circuit to generate intermediate kernel parameters, allowing parallel processing of convolutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If convolutions are performed using CPU, then neural network operations can be executed, but CPU bandwidth consumption increases and power consumption increases
Solution Approach 1:
The patent divides the neural network processing workload from the CPU into a dedicated neural processing unit (NPU). The NPU is segmented into specialized components including a data storage device for input data, a kernel storage device for convolution kernels, and a computation device with multipliers and accumulators. This segmentation allows the NPU to handle convolution operations independently, reducing CPU bandwidth consumption while maintaining neural network operation capability.
2Productivity
If convolutions are performed using CPU, then neural network operations can be executed, but power consumption increases
Solution Approach 1:
The patent replaces the general-purpose CPU mechanical system with a specialized neural processing unit optimized for convolution operations. The NPU uses dedicated hardware components (data storage device, kernel storage device, computation device with multipliers and accumulators) that are specifically designed for neural network computations. This substitution reduces power consumption by eliminating the overhead of general-purpose CPU execution while maintaining full neural network operation capability.
3Productivity
If Winograd transform is used for convolution, then computation efficiency improves, but device complexity increases
Solution Approach 1:
The patent applies the Winograd transform by pre-computing transformation matrices and storing them in the kernel storage device. The computation device uses these pre-computed transforms to perform convolutions more efficiently. While this adds some structural complexity to the NPU, the preliminary preparation of transformation data enables significant computation efficiency improvements, as the complex mathematical transformations are performed once and stored rather than recalculated for each convolution operation.
Data Source
AI summary
Embodiments relate to a neural processor circuit including a data storage device storing input data and a neural engine circuit. The input data includes a sequence of input parameters including a first subsequence of input parameters for a first pair of convolutions, a second subsequence of input parameters for a second pair of convolutions, and a third subsequence of input parameters for a third pair of convolutions. A kernel transformation circuit of the neural engine circuit can generate a number of intermediate kernel parameters. A first pair of accumulators can generate a first pair of convolution values at a first time instance based on the number of intermediate kernel parameters and generate a third pair of convolution values at a second time instance, while a second pair of accumulators can generate a second pair of convolution values at the first time instance based on the number of intermediate kernel parameters.