Neural Network Weight Quantization for Integer Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training neural networks using integer and/or fixed-point operators and registers result in poor performance during inference, as they require floating-point operations, which are computationally inefficient.
Innovation Solution
A method for training synaptic weights using quantized values with a specific quantization step and interval, allowing operations to be performed using integer and/or fixed-point operators and registers, maintaining performance during inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If quantization is applied to reduce memory footprint during training, then memory efficiency is improved, but inference performance deteriorates when using integer and/or fixed-point operators
Solution Approach 1:
The patent changes the quantization parameters by introducing a specific quantization step P that satisfies P ≤ 2^(-B-1) and restricts weight values to a predefined interval. This parameter adjustment allows the quantized weights to maintain sufficient precision for accurate inference while being representable in B bits, thus resolving the contradiction between memory efficiency and inference performance
Solution Approach 2:
The patent applies quantization during the training phase itself, rather than converting from floating-point after training. By performing quantization as a preliminary action during training with the specified quantization step and interval constraints, the network learns directly with quantized weights, ensuring both memory efficiency and inference performance are achieved simultaneously
2Reliability
If floating-point operators and registers are used for effective weight quantization, then inference performance is improved, but device complexity and computational efficiency worsen
Solution Approach 1:
The patent substitutes floating-point arithmetic operations with integer and/or fixed-point operations. By designing the quantization scheme with a specific step size and interval constraints, the patent enables accurate neural network inference using simpler integer arithmetic, replacing complex floating-point mechanics with more efficient integer-based computation
3Quantity of substance
If standard quantization methods are used, then memory footprint is reduced, but computation requirements increase for floating-point operations
Solution Approach 1:
The patent replaces energy-intensive floating-point computation with more efficient integer and/or fixed-point computation. The specific quantization design with constrained step size and interval allows accurate results to be achieved using less computationally expensive integer arithmetic operations, reducing energy consumption while maintaining memory efficiency
Data Source
AI summary
Method for learning synaptic weight values of a neural network, related data processing method, computer program, calculator and processing systemThe invention relates to a method for training synaptic weight values of at least one layer of an artificial neural network.The method is computer-implemented, and comprises training the weight values from training data, each weight value from said training being a quantized weight value belonging to a set of quantized values.The set of quantized values consists of values encoded with a predefined number B of bits, and with a quantization step P between two successive quantized values that satisfies:P=1⌊2B-12⌋where └·┘ represents the integer part function;the quantized values also being included in a predefined interval chosen from the interval [−1−P; 1] and the interval [−1; 1].