Z-First NPU Layout for Dual-Mode Winograd Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Neural Processing Units (NPUs) are designed primarily for direct convolution (DConv) methods and struggle to efficiently implement Winograd convolution (WgConv) due to differences in data layouts and computational requirements, leading to suboptimal performance.
Innovation Solution
A z-first reference neural processing unit (NPU) is designed with memory banks configured for both DConv and WgConv modes, featuring a reconfigurable IFM distributor, parallel reconfigurable Winograd forward transform module, MAC units, and a reconfigurable OFM adder and inverse transform module, enabling efficient transformation and dot product operations for both modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing NPUs are designed primarily for direct convolution (DConv) methods, then DConv performance is optimized, but Winograd convolution (WgConv) implementation efficiency deteriorates
Solution Approach 1:
The NPU is designed with dual-mode support for both DConv and WgConv operations. The architecture includes configurable data layout options (NHWC and NCHW formats) and reconfigurable computational units that can adapt to different convolution algorithms, allowing a single device to efficiently execute multiple convolution types without requiring separate specialized hardware for each method.
Solution Approach 2:
The NPU employs dynamic reconfiguration capabilities where the data layout format (NHWC or NCHW) and operational mode (DConv or WgConv) can be changed at runtime based on the specific computational requirements. This dynamic adaptability allows the system to optimize performance for different convolution algorithms by adjusting its internal configuration to match the algorithm's data access patterns and computational structure.
2Device complexity
If NPUs use fixed data layouts, then hardware design is simplified, but efficiency for different convolution algorithms deteriorates
Solution Approach 1:
The NPU implements dynamic data layout switching between NHWC and NCHW formats through reconfigurable memory access patterns and data permutation units. This allows the hardware to adapt its data organization to match the requirements of different convolution algorithms (DConv or WgConv) without requiring multiple fixed-layout hardware instances, thereby maintaining reasonable design complexity while achieving high algorithm efficiency.
Solution Approach 2:
The system changes key parameters including data layout format (NHWC/NCHW), block size configurations, and memory access strides based on the selected convolution algorithm. These parameter changes enable the same hardware infrastructure to efficiently support both DConv and WgConv by adjusting data organization and access patterns to match each algorithm's optimal performance characteristics.
3Productivity
If Winograd convolution is implemented on DConv-optimized NPUs, then algorithmic speedup is achieved, but data layout transformation overhead increases
Solution Approach 1:
The NPU performs data layout transformation and Winograd domain conversion in advance before the main convolution computation. By pre-processing the input feature maps and kernels into the appropriate Winograd domain format using dedicated transform units, the system eliminates the need for expensive runtime transformations during convolution execution, thereby achieving the algorithmic speedup of WgConv while minimizing transformation overhead.
Solution Approach 2:
The architecture introduces intermediary transform units that convert data between NHWC and NCHW formats and between spatial and Winograd domains. These intermediary components act as dedicated buffers and transformation engines that handle the data layout conversion efficiently, separating the transformation overhead from the main convolution computation and enabling smooth transitions between different convolution algorithms without significant performance penalty.
Data Source
AI summary
A z-first reference neural processing unit (NPU) for mapping Winograd Convolution is disclosed where the NPU includes memory banks configured to store input feature maps (IFMs) in a z-first data storage layout, each of the memory banks being configured to store the IFMs in one of a direct convolution (DConv) mode or a Winograd convolution (WgConv) mode, a reconfigurable IFM distributor configured to receive the IFMs from the memory banks, a parallel reconfigurable Winograd forward transform module configured to receive the IFMs from the reconfigurable IFM distributor and to transform the IFMs in a Winograd domain to transformed IFMs in the WgConv mode, multiply and accumulate (MAC) units configured to perform dot product operations on one of IFMs in the DConv mode and the transformed IFMs in the WgConv mode to obtain intermediate output feature maps (OFMs), and a reconfigurable OFM adder and Winograd inverse transform module configured to generate one of an OFM from the intermediate OFMs in the DConv mode and OFMs from the intermediate OFMs in the WgConv.