Feature Map Quantization Around Activation Asymptotes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for processing deep neural networks (DNNs) face challenges in managing memory capacity and computational load due to varying feature map sizes, leading to increased memory requirements and processing complexity, especially when feature maps are spatially divided.
Innovation Solution
An information processing device and method that derives and encodes differences between feature maps and asymptotic activation function values using quantization-based methods, avoiding zero input quantization levels to suppress data size increases and minimize distortion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If feature maps are spatially divided to suppress memory capacity increase, then memory requirements are reduced, but computational load and processing complexity increase
Solution Approach 1:
The patent applies parameter changes by transforming the feature map data through quantization to reduce its data size. By changing the representation parameters of the feature map (from full precision to quantized levels), the system reduces memory requirements without needing to spatially divide the data, thereby avoiding the associated processing complexity increase.
2Volume of stationary object
If feature maps are spatially divided and processed separately, then memory capacity requirements are suppressed, but processing time increases
Solution Approach 1:
The patent uses parameter changes through quantization to reduce feature map data size, enabling the system to process feature maps without spatial division. This approach maintains processing time efficiency while reducing memory capacity requirements, as the quantized feature maps can be handled in-place without the time overhead of division and recombination operations.
3Quantity of substance
If quantization is applied to encode feature maps, then data size is reduced, but distortion in feature map values increases
Solution Approach 1:
The patent optimizes the quantization parameters by setting the zero quantization level to match the asymptotic value of the activation function. This parameter adjustment ensures that the quantization process minimizes distortion in feature map values, particularly near the asymptotic boundaries, while still achieving significant data size reduction through quantization.
Solution Approach 2:
The patent replaces traditional spatial division methods with a parameter-based quantization approach. Instead of mechanically dividing the feature map into spatial regions, the system substitutes this with a parameter transformation that encodes the feature map values into quantized levels, achieving data size reduction without the associated losses of spatial division.
Data Source
AI summary
The present disclosure relates to an information processing device and method that can suppress an increase in data size of a feature map. A difference between a feature map that is a processing result of a computational layer subject to processing of a neural network and an asymptotic value of an activation function of the computational layer subject to processing is derived, and the difference is encoded by a quantization-based method where a midpoint of a quantization step size is not set as a quantization level for zero input. Furthermore, the encoded data is decoded to generate the difference between the feature map and the asymptotic value, and the feature map is derived using the difference and the asymptotic value. The present disclosure is applicable to, for example, an information processing device, an image processing device, an electronic device, an information processing method, an image processing method, a program, or the like.