Per-Channel Affine Transformation Neural Network Accelerator

Solution Overview

Problem

Deep Neural Networks (DNNs) are computationally expensive to implement, especially when processing network parameters in floating point formats, which increases silicon area, power consumption, and complexity, while reducing them to fewer bits for efficiency compromises accuracy.

Innovation Solution

Implementing a per-channel quantised affine transformation on a neural network accelerator using a depth-wise convolution processing unit for additions and a high-precision re-quantisation processing unit for multiplications, allowing efficient and accurate operations without floating point multiplications.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If floating point formats are used for network parameters, then accuracy is maintained, but silicon area, power consumption, and complexity increase

Engineering Contradiction:

ImproveaccuracyVSAvoidhardware complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent changes the numerical representation parameter from floating point to quantised integer formats. By representing network parameters (weights and biases) as quantised values with associated scale factors, the system maintains computational accuracy while enabling the use of simpler integer arithmetic units instead of complex floating point units, thus reducing hardware complexity.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent segments the representation of network parameters into two parts: quantised integer values and scale factors. This segmentation allows the system to perform calculations using simple integer arithmetic for the quantised values while separately managing the scale factors, thereby avoiding the need for complex floating point hardware while maintaining precision.

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If floating point formats are used for network parameters, then accuracy is maintained, but power consumption increases

Engineering Contradiction:

ImproveaccuracyVSAvoidpower consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent changes the numerical representation parameter from floating point to quantised integer formats. By representing network parameters (weights and biases) as quantised values with associated scale factors, the system maintains computational accuracy while enabling the use of simpler integer arithmetic units instead of complex floating point units, thus reducing hardware complexity.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent uses simpler quantised integer representations instead of complex floating point representations. The quantised values with scale factors act as a cheaper alternative to floating point hardware, providing sufficient accuracy for neural network computations while significantly reducing power consumption through the use of simpler arithmetic operations.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

3Productivity

If fewer bits are used for network parameters, then efficiency increases, but accuracy is compromised

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidaccuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent changes the numerical representation parameter from floating point to quantised integer formats. By representing network parameters (weights and biases) as quantised values with associated scale factors, the system maintains computational accuracy while enabling the use of simpler integer arithmetic units instead of complex floating point units, thus reducing hardware complexity.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent adds a new dimension to the representation by introducing scale factors alongside quantised integer values. This dimensional addition allows the system to compensate for the reduced precision of fewer-bit quantised values, effectively maintaining accuracy while achieving efficiency gains from the reduced bit width of the primary parameter representation.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS20240169191A1Methods and systems for performing a per channel affine transformation using a neural network accelerator

Publication Date: 2024.05.23 IMAGINATION TECH LTD

AI summary

Neural network accelerators implement per channel quantised affine transformation on an input tensor to generate an output tensor, the affine transformation for a channel comprising a multiplication by a multiplication parameter followed by an addition of an addition parameter. A depth-wise convolution processing unit accelerates depth-wise convolution operations, and a high-precision re-quantisation processing unit accelerates re-quantisation operations. Each affine transformation is implemented as an addition followed by a multiplication by: implementing the additions by performing a 1×1 depth-wise convolution on the input tensor based on a weight and a bias for each channel of the input tensor, wherein the weight for a channel is set to an addition scale factor for the channel and the bias for a channel is set to an integer addition value for the channel; and implementing the multiplications by scaling, using the high-precision re-quantisation processing unit, each value output from the depth-wise convolution processing unit, by a high-precision multiplication value for the corresponding channel.