Neural Network Data Quantizing via Local and Global Maximum Values
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Typical neural network apparatuses face challenges in efficiently processing large amounts of data in real-time, especially in low-power and high-performance systems like smartphones, due to inefficiencies in network operations.
Innovation Solution
A method and apparatus for neural network data quantizing that involves obtaining local and global quantization data by quantizing output data based on maximum values, allowing for efficient storage and use in subsequent layers, thereby improving real-time data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If typical neural network apparatus performs large amount of computations to process complicated input data, then processing accuracy is improved, but processing speed and real-time performance deteriorate
Solution Approach 1:
The patent applies parameter changes by transforming neural network data from floating-point format to fixed-point quantized format. This changes the numerical representation parameters, enabling faster computation while maintaining acceptable accuracy. The quantization process converts continuous floating-point values to discrete fixed-point values, reducing computational complexity and improving processing speed for real-time applications.
Solution Approach 2:
The patent implements local quality by applying different quantization strategies to different parts of the neural network data. Specifically, it uses channel-wise quantization where each channel has its own scale factor and quantization parameters, rather than uniform quantization across all channels. This localized approach maintains higher accuracy for important channels while optimizing overall processing efficiency.
2Use of energy by moving object
If neural network apparatus processes data in real-time on low-power systems, then energy efficiency is improved, but computational capability and processing performance worsen
Solution Approach 1:
The patent changes the numerical representation parameter from floating-point to fixed-point, which significantly reduces the computational resources required. Fixed-point arithmetic requires fewer transistors and less power consumption compared to floating-point operations, making it suitable for low-power embedded systems while maintaining real-time processing capability.
Solution Approach 2:
The patent uses simplified fixed-point quantized data structures that are computationally cheaper and require less hardware resources. These quantized representations act as simplified substitutes for full-precision floating-point data, enabling execution on resource-constrained devices with limited power supply and computational capability.
3Device complexity
If global quantization is applied to all output channels, then device complexity is reduced, but measurement precision and data accuracy worsen
Solution Approach 1:
The patent applies local quality by implementing channel-wise quantization where each output channel has its own quantization parameters (scale factors and maximum values) calculated independently. This allows the system to adapt to the specific characteristics of each channel, maintaining higher accuracy for channels with different data distributions while avoiding the oversimplification of global quantization.
Solution Approach 2:
The patent segments the quantization process by channel, dividing the output data into separate channels and applying individual quantization to each. This segmentation enables precise control over quantization parameters per channel, balancing the trade-off between complexity and accuracy by allowing fine-grained optimization without requiring complex per-element quantization.
4Quantity of substance
If quantization is performed to reduce memory usage, then resource efficiency is improved, but information loss and data precision worsen
Solution Approach 1:
The patent changes the data representation parameter from high-precision floating-point to lower-precision fixed-point quantized format. This parameter change reduces the memory storage requirements significantly while attempting to preserve essential information through careful selection of quantization parameters and scale factors that minimize information loss.
Solution Approach 2:
The patent applies local quality by using channel-specific scale factors and quantization ranges that are optimized to preserve the most important information in each channel. This localized optimization ensures that critical data characteristics are maintained even with reduced precision, minimizing information loss while achieving memory efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A neural network data quantizing method includes: obtaining local quantization data by firstly quantizing, based on a local maximum value for each output channel of a current layer of a neural network, global recovery data obtained by recovering output data of an operation of the current layer based on a global maximum value corresponding to a previous layer of the neural network; storing the local quantization data in a memory to perform an operation of a next layer of the neural network; obtaining global quantization data by secondarily quantizing, based on a global maximum value corresponding to the current layer, local recovery data obtained by recovering the local quantization data based on the local maximum value for each output channel of the current layer; and providing the global quantization data as input data for the operation of the next layer.