Depthwise Convolution NPU Core Layout for Higher Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural processing units (NPUs) exhibit low utilization rates when performing depthwise convolution operations due to inefficient data path configurations, leading to underutilization and inefficiency, particularly with vector processors remaining idle during normal convolution operations.
Innovation Solution
The data paths of activations and weights are inverted within the NPU core, and 2-to-1 multiplexers are added for every 2/9 multipliers, operating the core using a rows x columns and cells (RSxHW) dataflow to maximize spatial feature map locality and improve NPU utilization for depthwise convolutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional NPU cores are designed for normal convolution operations, then normal convolution performance is optimized, but depthwise convolution utilization rate deteriorates
Solution Approach 1:
The patent implements dynamic data path configuration where the NPU core can switch between normal convolution mode and depthwise convolution mode. In depthwise convolution mode, the system dynamically reconfigures data paths to process input channels along the depth dimension rather than reducing products in the input channel direction, thereby optimizing utilization for depthwise convolution operations while maintaining normal convolution capability
Solution Approach 2:
The patent changes key parameters of the NPU core configuration, including the unrolling of input channels and output channels in space, and modifies the reduction operation from dot-product along input channel direction to processing along depth dimension. This parameter change enables the same hardware to efficiently execute both normal and depthwise convolution operations
2Productivity
If a dedicated vector processor is attached to the NPU core to solve low utilization, then depthwise convolution performance improves, but device complexity and load balance deteriorate
Solution Approach 1:
The patent makes the NPU core itself universal by enabling it to perform both normal convolution and depthwise convolution operations through reconfigurable data paths. Instead of adding a separate dedicated vector processor, the same NPU core units are reconfigured to handle depthwise convolution, eliminating the need for additional specialized hardware and reducing overall device complexity
Solution Approach 2:
The patent merges the functionality of normal convolution and depthwise convolution into a single NPU core architecture. By combining both operation types in one core rather than separating them into different processors, the system achieves better load balance and reduces complexity while maintaining high performance for both convolution types
3Productivity
If data paths are inverted and RSxHW dataflow is used, then spatial feature map locality is maximized, but data path configuration complexity increases
Solution Approach 1:
The patent segments the data path into distinct components with specific functions: input channels are unrolled in space, output channels are unrolled in space, and multiplexers are strategically placed to route data along the RSxHW dataflow pattern. This segmentation allows the complex data path to be managed through modular, well-defined sections that work together to maximize spatial locality
Data Source
Figure 1~2B
Figure 3A~3B
Figure 4A~4B
AI summary
A core of neural processing units is configured to efficiently process a depthwise convolution by maximizing spatial feature-map locality using adder trees. Data paths of activations and weights are inverted, and 2-to-1 multiplexers are every 2/9 multipliers along a row of multipliers. During a depthwise convolution operation, the core is operated using a RSxHW dataflow to maximize the locality of feature maps. For a normal convolution operation, the data paths of activations and weights may be configured for a normal convolution configuration and in which multiplexers are idle.