Neural Network Parameter Quantization via Multiplier and Bit Shift

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Neural networks require significant computational resources due to the large number of parameters and complex dot product operations, necessitating efficient encoding and decoding methods to reduce bitstream size and computational complexity.

Innovation Solution

The proposed solution involves quantizing neural network parameters using a quantization parameter and value, determining a multiplier and bit shift number based on division operations, allowing for integer domain computations and reducing the need for floating-point operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If neural network parameters are represented using standard floating-point formats, then computational accuracy is maintained, but bitstream size and storage requirements increase significantly

Engineering Contradiction:

Improvecomputational accuracyVSAvoidbitstream size

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent changes the representation parameters of neural network weights by introducing a factorized format where each weight is represented as a product of a codebook entry and a scale factor. This parameter transformation enables more compact representation while maintaining computational accuracy through the mathematical equivalence of the factorized form.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent segments the weight representation into two separate components: a codebook index (selecting from predefined values) and a scale factor (adjusting the magnitude). This segmentation allows independent optimization of each component, achieving better compression ratios while preserving the ability to represent a wide range of weight values accurately.

Inventive Principle:

Principle #1Segmentation

2Quantity of substance

If neural network parameters are quantized to reduce bitstream size, then storage and transmission efficiency improve, but computational complexity and quantization errors increase

Engineering Contradiction:

Improvebitstream sizeVSAvoidquantization error

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

The patent introduces dynamic scale factors that can be adjusted based on the specific weight being represented. Rather than using fixed quantization steps, the scale factor adapts to the magnitude and distribution of weights, allowing finer resolution where needed and coarser representation where acceptable, thereby reducing overall quantization error.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The codebook serves as an intermediary structure between the original continuous weights and their quantized representations. By selecting from pre-defined codebook entries and applying scale factors, the system achieves accurate approximation of original weights while maintaining compact representation, effectively mediating between precision and compression requirements.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Quantity of substance

If neural network parameters are compressed using traditional quantization methods, then storage requirements decrease, but inference accuracy and model performance deteriorate

Engineering Contradiction:

Improvestorage requirementsVSAvoidinference accuracy

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The patent creates a composite representation format combining discrete codebook entries with continuous scale factors. This composite approach leverages the benefits of both discrete quantization (compact storage) and continuous representation (high precision), achieving a balance that maintains inference accuracy while significantly reducing storage requirements compared to full-precision floating-point formats.

Inventive Principle:

Principle #40Composite materials

4Measurement precision

If high-precision floating-point representations are used for neural network parameters, then model accuracy is preserved, but computational resources and processing time increase

Engineering Contradiction:

Improvemodel accuracyVSAvoidcomputational efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent substitutes traditional floating-point arithmetic operations with simplified computations based on codebook indices and scale factors. Instead of performing full floating-point multiplication and addition, the system uses pre-computed codebook entries combined with simple scaling operations, significantly reducing computational complexity while maintaining model accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20230075514A1Concept for a representation of neural network parameters

Publication Date: 2023.03.09 FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV

AI summary

Apparatus for generating a NN representation, configured to quantize an NN parameter onto a quantized value by determining a quantization parameter and a quantization value for the NN parameter so that from the quantization parameter, there is derivable a multiplier and a bit shift number. Additionally, the determining of the quantization parameter and the quantization value for the NN parameter is performed so that the quantized value of the NN parameter corresponds to a product between the quantization value and a factor, which depends on the multiplier, bit-shifted by a number of bits which depends on the bit shift number.