Quantization Aware Training Input Constraint Neural Network

Solution Overview

Problem

Conventional quantization aware training methods for neural networks often result in reduced inference accuracy on edge devices due to quantization losses, particularly in convolution layers, as they do not adequately account for constraints such as concat operations and transcendental functions, leading to suboptimal performance.

Innovation Solution

The method involves modifying the neural network model by quantizing the inputs of convolution layers instead of outputs, allowing all layers to maintain higher precision outputs and using integrated circuits to optimize performance for edge devices, thereby minimizing performance impact and enhancing accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If quantization aware training is implemented with conventional methods, then training process is simplified, but inference accuracy on edge devices deteriorates due to unmodeled constraints

Engineering Contradiction:

Improvetraining process simplicityVSAvoidinference accuracy

Core Design Contradiction:

Ease of manufactureVSMeasurement precision

Solution Approach 1:

The patent applies preliminary action by constraining the input data to specific ranges before it enters the convolutional layers during training. This pre-constraint ensures that the neural network learns to handle quantized data properly from the beginning, accounting for the limited precision and range of fixed-point arithmetic before any computations occur, thereby resolving the accuracy deterioration issue while maintaining training simplicity

Inventive Principle:

Principle #10Preliminary action

2Power

If lower container format is used for edge devices, then computational cost and bandwidth are reduced, but precision and inference accuracy deteriorate

Engineering Contradiction:

Improvecomputational costVSAvoidinference accuracy

Core Design Contradiction:

PowerVSMeasurement precision

Solution Approach 1:

The patent applies parameter changes by modifying the input data parameters (range and precision) to match the constraints of the lower container format before feeding it into the neural network. This transformation ensures that the network receives data in the appropriate format for fixed-point arithmetic, allowing edge devices to operate efficiently with lower precision while maintaining acceptable inference accuracy through proper data preparation

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If quantization loss is minimized in convolution layers, then inference accuracy improves, but constraints from concat and transcendental operations are not accounted for

Engineering Contradiction:

Improveinference accuracyVSAvoidconstraint modeling capability

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent applies preliminary action by constraining the input data to specific ranges before it enters the convolutional layers during training. This pre-constraint ensures that the neural network learns to handle quantized data properly from the beginning, accounting for the limited precision and range of fixed-point arithmetic before any computations occur, thereby resolving the accuracy deterioration issue while maintaining training simplicity

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11615304B1Quantization aware training by constraining input

Publication Date: 2023.03.28 AMBARELLA INT LP

AI summary

A method of generating a quantized neural network comprises (i) receiving a neural network model, (ii) modifying the neural network model by quantizing input of at least convolution layers of the neural network model, and (iii) training the modified neural network model using a dataset that is representative of one or more desired inferences.