NPU PE Array Weight Reuse for Depth-Wise Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing neural processing units (NPUs) face inefficiencies in depth-wise convolution operations due to low utilization rates of processing element arrays, leading to increased power consumption and processing time, despite requiring fewer computations compared to standard convolution operations.
Innovation Solution
The NPU is designed to reuse weights during depth-wise convolution operations by employing a mode selector that switches between first and second modes, allowing for pipelined data transfer and broadcast of input data, and includes a delay buffer to optimize processing element array efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weights are read from main memory for each depth-wise convolution operation, then the operation can be performed, but the number of read operations increases and power consumption increases
Solution Approach 1:
The patent loads weights from main memory into the processing element array before performing depth-wise convolution operations. By pre-loading the weights into on-chip memory, the system eliminates the need for repeated main memory reads during the convolution process, thereby reducing power consumption while maintaining processing throughput.
Solution Approach 2:
The processing element array is designed to handle both standard convolution and depth-wise convolution operations using the same hardware resources. By reusing the loaded weights across multiple processing elements and operations, the system maximizes resource utilization and reduces redundant memory access, addressing both productivity and energy efficiency.
2Productivity
If processing element array is used for depth-wise convolution, then computation can be performed, but utilization rate is low
Solution Approach 1:
The patent divides the processing element array into multiple groups or channels, where each group can independently process different feature maps or convolution kernels. This segmentation allows parallel processing of multiple depth-wise convolution operations simultaneously, increasing the utilization rate of the processing element array while maintaining computation capability.
Solution Approach 2:
The patent introduces an additional dimension of parallelism by allowing multiple processing elements to share and reuse the same weight data across different computation streams. This dimensional expansion enables the system to fully utilize the processing element array for depth-wise convolution operations without requiring proportional increases in computation complexity.
Data Source
AI summary
A neural processing unit includes a mode selector configured to select a first mode or a second mode; and processing element (PE) array operating in one of the first mode and the second mode and including a plurality of processing elements arranged in PE rows and PE columns, the PE array configured to receive an input of first input data and an input of second input data, respectively. In the second mode, the first input data is inputted in a PE column direction of the PE array and is transmitted along the PE column direction while being delayed by a specific number of clock cycles, and the second input data is broadcast to the plurality of processing elements of the PE array to which the first input data is delayed by the specific number of clock cycles.