Neural Network Activation Data Bit Width Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing methods are inefficient in reducing bit width for activation data, leading to high computational resource usage and processing time, especially in resource-constrained environments like mobile terminals.
Innovation Solution
A method that generates output maps of a neural network's current layer by performing convolution operations, determines a lightweight format based on activation data distribution, and shifts data to achieve low bit width, allowing for adaptive bit width reduction in both input and output maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional neural network processing is used without bit width reduction, then processing accuracy is maintained, but computational resource usage and processing time increase significantly
Solution Approach 1:
The patent changes the bit width parameter of activation data from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integer) during neural network processing. This parameter transformation reduces computational resource usage and increases processing speed while maintaining acceptable accuracy through careful selection of quantization strategies and scaling factors.
Solution Approach 2:
The patent uses lightweight, low-precision data representations instead of heavy, high-precision data structures. By employing low-bit width activation data that requires less computational overhead and memory bandwidth, the system achieves faster processing with reduced energy consumption, effectively replacing expensive computational operations with cheaper alternatives.
2Productivity
If bit width reduction is applied to activation data, then computational efficiency improves, but additional operations are required for format conversion
Solution Approach 1:
The patent performs bit width reduction and format conversion operations in advance, before the main computational workload. By quantizing activation data to low bit width early in the processing pipeline and maintaining this format throughout subsequent layers, the system avoids repeated format conversions and reduces the overall complexity of processing operations.
Solution Approach 2:
The patent maintains continuous use of low-bit width data format throughout the neural network processing pipeline. Once activation data is converted to low bit width, this format is preserved across multiple layers and operations, eliminating the need for repeated high-precision conversions and enabling sustained computational efficiency gains.
3Quantity of substance
If low bit width format is used for activation data, then memory usage is reduced, but precision and accuracy may be compromised
Solution Approach 1:
The patent carefully manages the precision parameter by using low bit width formats (e.g., 8-bit integers) for activation data while employing appropriate scaling factors and quantization strategies. This approach significantly reduces memory usage compared to high-precision formats while maintaining sufficient accuracy for practical neural network applications through optimized parameter selection.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processing method using a neural network includes generating output maps of a current layer of the neural network by performing a convolution operation between input maps of the current layer and weight kernels of the current layer, determining a lightweight format for the output maps of the current layer based on a distribution of at least a portion of activation data being processed in the neural network, and lightening activation data corresponding to the output maps of the current layer to have a low bit width based on the determined lightweight format.