Deep Neural Network Activation Volume Quantization via Depth Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks face high computational complexity and bandwidth requirements due to large data operations, and direct global quantization of activation volumes leads to significant quantization errors due to varying element values across layers.
Innovation Solution
The method divides activation volumes into depth segments with smaller value differences, creating slice groups that are quantized using unique quantization parameters, reducing the need for a single quantization parameter across all elements and minimizing quantization errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If global quantization is performed on activation volume using a single quantization parameter, then the quantization process is simple and fast, but large quantization errors occur due to the large difference among elements in the activation volume
Solution Approach 1:
The activation volume is divided into multiple depth segments along the depth direction, where each segment contains elements with similar value ranges. This segmentation allows different quantization parameters to be applied to different segments, improving quantization accuracy without significantly increasing computational complexity.
Solution Approach 2:
Each depth segment is assigned its own quantization parameter (such as quantization step size) based on the local characteristics of elements within that segment. This local adaptation ensures that elements with similar values are quantized with appropriate precision, reducing overall quantization error.
2Measurement precision
If the activation volume is divided into multiple slice groups with different quantization parameters, then quantization accuracy improves, but the device complexity and computational overhead increase
Solution Approach 1:
The activation volume is divided into multiple depth segments along the depth direction, where each segment contains elements with similar value ranges. This segmentation allows different quantization parameters to be applied to different segments, improving quantization accuracy without significantly increasing computational complexity.
Solution Approach 2:
Each depth segment is assigned its own quantization parameter (such as quantization step size) based on the local characteristics of elements within that segment. This local adaptation ensures that elements with similar values are quantized with appropriate precision, reducing overall quantization error.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
A method and an apparatus for quantizing an activation volume of a deep neural network are disclosed. The method includes: obtaining an activation volume of a network layer in the deep neural network (S101), wherein elements in the activation volume are arranged in three directions: height, width, and depth; dividing depth segments in the activation volume in which a difference among element features is smaller than a preset threshold into a same slice group along the depth direction of the activation volume, so as to obtain a plurality of slice groups (SI02); quantizing each slice group respectively by using a quantization parameter corresponding to each slice group obtained through a quantization formula (S103). The quantization error can be reduced through the above method.