Mixed-Precision NPU Tile for Depth-Wise Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processors lack support for depth-wise separable convolution and direct convolution on image data stored in planar-wise order, and they do not efficiently handle activations and weights with zero or near-zero values, leading to suboptimal power consumption and silicon area usage.
Innovation Solution
A mixed-precision neural processor tile with depth-wise convolution capabilities, featuring a shuffler, multipliers, accumulators, and bit shifters, that can perform convolutions on tensors with zero or near-zero values, and efficiently handle image data in planar-wise order, optimizing power consumption and silicon area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the NPU is designed to perform direct convolution on depth-major ordered tensors, then convolution computation capability is improved, but support for depth-wise separable convolution and planar-wise ordered image data is lost
Solution Approach 1:
The NPU architecture is designed to be dynamically reconfigurable, allowing it to switch between different convolution modes (direct convolution, depth-wise separable convolution) and data ordering formats (depth-major, planar-wise). This enables the same hardware to adapt to different computational requirements without being locked into a single configuration.
Solution Approach 2:
The NPU is designed with universal functionality to perform multiple types of convolution operations (direct convolution, depth-wise separable convolution) and handle different data ordering formats. By incorporating flexible data reordering mechanisms and configurable computational units, the NPU can serve multiple purposes within a single architecture.
2Measurement precision
If the NPU processes all activation and weight values with full precision, then computation precision is maintained, but power consumption and silicon area increase
Solution Approach 1:
The NPU implements mixed-precision computation by applying different precision levels to different parts of the computation. Specifically, activations and weights that are frequently zero or near-zero are processed with lower precision (e.g., 4-bit or 8-bit), while other values maintain higher precision. This localized precision optimization reduces overall power consumption and silicon area while maintaining adequate computation precision.
Solution Approach 2:
The NPU dynamically changes precision parameters based on the characteristics of the input data. When activations or weights are detected to have frequent zero or near-zero values, the system adjusts the precision parameter downward for those specific operations, thereby reducing power consumption and silicon area requirements while maintaining sufficient computation precision.
3Area of stationary object
If the NPU uses a simplified architecture to reduce silicon area, then power consumption decreases, but support for diverse convolution operations and data formats is reduced
Solution Approach 1:
The NPU architecture is segmented into modular functional units that can be selectively activated based on the required operation. By dividing the NPU into independent modules (e.g., separate handling paths for direct convolution and depth-wise separable convolution), the system can activate only the necessary segments for each operation, reducing the effective silicon area in use while maintaining full versatility.
Solution Approach 2:
The NPU employs dynamic resource allocation where computational units and data paths are activated or deactivated based on the specific operation being performed. This dynamic approach allows the NPU to maintain full versatility across different convolution types while using only the necessary silicon area for each particular operation, rather than dedicating hardware for all possible operations simultaneously.
Data Source
AI summary
A processor to perform inference on deep learning neural network models. In some embodiments, the process includes: a first tile, a second tile, a memory, and a bus, the bus being connected to: the memory, the first tile, and the second tile, the first tile including: a first weight register, a second weight register, an activations cache, a shuffler, an activations buffer, a first multiplier, and a second multiplier, the activations buffer being configured to include: a first queue connected to the first multiplier, and a second queue connected to the second multiplier, the activations cache including a plurality of independent lanes, each of the independent lanes being randomly accessible, the first tile being configured: to receive a tensor including a plurality of two-dimensional arrays, each representing one color component of the image; and to perform a convolution of a kernel with one of the two-dimensional arrays.