Offset Function for Neural Network Weight Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Combining pruning and quantization techniques in neural networks results in highly sparse, quantized networks with reduced accuracy, which is sub-optimal for mobile and resource-constrained devices.
Innovation Solution
A method that combines dynamic magnitude pruning with low-bitwidth training methodologies and a modified number system using an offset function to generate and quantify weight values, allowing for efficient storage and computation while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If pruning and quantization techniques are combined to reduce neural network complexity, then storage and computational resources are reduced, but accuracy deteriorates
Solution Approach 1:
The patent changes the parameter representation by introducing an offset function that transforms weight values into a different numerical range before quantization. This parameter transformation allows the quantized weights to maintain better accuracy while still achieving compression, directly resolving the contradiction between reduced complexity and maintained precision.
Solution Approach 2:
The offset function serves as an intermediary mechanism between the original weight values and the quantized representation. By introducing this intermediate transformation step, the patent enables accurate representation of weights in a compressed format, thereby reducing network complexity without sacrificing accuracy.
2Quantity of substance
If low-bitwidth quantization is applied to neural network weights, then storage requirements are reduced, but computational accuracy deteriorates
Solution Approach 1:
The patent applies parameter changes by transforming weight values through an offset function that maps them to a suitable range for low-bitwidth quantization. This transformation ensures that even with limited bits for storage, the quantized values maintain sufficient precision for accurate computation, thus reducing storage requirements while preserving computational accuracy.
3Use of energy by moving object
If highly sparse quantized neural networks are created, then resource consumption is reduced, but performance deteriorates
Solution Approach 1:
The offset function transforms weight parameters into a format that enables effective low-bitwidth quantization while maintaining performance. By changing the parameter representation, the patent achieves sparse quantized networks that consume fewer resources but maintain reliable performance through the improved quantization scheme.
Data Source
AI summary
Neural network systems and methods are provided. One method for processing a neural network includes, for at least one neural network layer that includes a plurality of weights, applying an offset function to each of a plurality of weight values in the plurality of weights to generate an offset weight value, and quantizing the offset weight values to form quantized offset weight values. The plurality of weights are pruned. One method for executing a neural network includes reading, from a memory, at least one neural network layer that includes quantized offset weight values and an offset value α, and performing a neural network layer operation on an input feature map, based on the quantized offset weight values and the offset value α, to generate an output feature map. The quantized offset weight values are signed integer numbers.