Neural Processing Core Reconfiguration for Depthwise Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural processing units (NPUs) exhibit low utilization rates when performing depthwise convolution operations due to their optimal design for normal convolution operations, leading to inefficiencies and underutilization of resources.
Innovation Solution
The NPU core is reconfigured with an array of processing elements (PEs) arranged in specific row groups and multiplexers to optimize data paths for depthwise convolutions, utilizing a RS×HW dataflow and adder trees to maximize spatial feature-map locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the NPU core is configured for normal convolution operations, then the NPU achieves optimal performance for normal convolutions, but the NPU utilization rate drops to approximately 3% when performing depthwise convolution operations
Solution Approach 1:
The patent implements dynamic reconfiguration of the NPU core by introducing multiplexers that can switch between different operational modes (normal convolution and depthwise convolution). The multiplexers dynamically route data paths based on the operation type, allowing the same hardware to adapt its behavior without physical reconfiguration. This resolves the contradiction by enabling the NPU to maintain optimal performance for normal convolutions while also achieving high utilization for depthwise convolutions through dynamic mode switching.
Solution Approach 2:
The patent makes the NPU core universal by enabling it to perform both normal convolution and depthwise convolution operations using the same hardware architecture. The multiplexer-based reconfiguration allows a single NPU core to serve multiple functions, eliminating the need for separate dedicated hardware for different convolution types. This multi-functionality directly addresses the adaptability issue while maintaining high productivity for both operation types.
2Adaptability or versatility
If a dedicated vector processor is attached to the NPU core to handle depthwise convolutions, then the NPU can perform depthwise convolutions, but the vector processor remains idle during normal convolution operations, reducing overall chip efficiency
Solution Approach 1:
Instead of adding a separate dedicated vector processor that would remain idle during normal convolutions, the patent makes the existing NPU core universal by enabling it to perform both normal and depthwise convolutions through multiplexer-based reconfiguration. This eliminates the need for additional dedicated hardware, ensuring that all processing units remain actively utilized for both operation types, thereby maintaining overall chip efficiency while gaining depthwise convolution capability.
Solution Approach 2:
The patent merges the functionality of normal convolution and depthwise convolution into a single unified NPU core architecture. By combining both operation types into one hardware structure with multiplexer-controlled data paths, the system eliminates the need for separate processors. This merging ensures that all processing resources are actively engaged in useful work for both convolution types, maximizing overall chip productivity.
3Productivity
If the NPU core is reconfigured with multiplexers and adder trees to maximize spatial feature-map locality, then the NPU utilization for depthwise convolutions improves significantly, but the device complexity increases
Solution Approach 1:
The patent segments the NPU core into modular units with specific multiplexer configurations for different row groups (RG1-RG3). Each row group has a standardized multiplexer pattern that can be independently configured, making the complex reconfiguration task manageable through modular segmentation. This segmentation approach reduces the overall complexity by breaking down the reconfiguration problem into smaller, repeatable units that follow the same design template.
Solution Approach 2:
The patent achieves high NPU utilization for depthwise convolutions by changing operational parameters (data paths, multiplexer settings, adder tree configurations) rather than fundamentally redesigning the hardware. The same physical architecture supports both operation types through parameter changes in the multiplexer routing and computational graph configuration. This parameter-based reconfiguration improves productivity while minimizing the increase in device complexity compared to hardware redesign.
Data Source
AI summary
A core of neural processing units is configured to efficiently process a depthwise convolution by maximizing spatial feature-map locality using adder trees. Data paths of activations and weights are inverted, and 2-to-1 multiplexers are every 2/9 multipliers along a row of multipliers. During a depthwise convolution operation, the core is operated using a RS×HW dataflow to maximize the locality of feature maps. For a normal convolution operation, the data paths of activations and weights may be configured for a normal convolution configuration and in which multiplexers are idle.