Mixed-Format Neural Network Quantization for Accuracy and Power

Solution Overview

Problem

Conventional neural network technologies face challenges in simultaneously increasing both accuracy and dynamic range during the quantization of floating-point type input data and weights, often degrading one aspect to improve the other.

Innovation Solution

A neural network device and method that quantizes M-bit floating-point type parameters into N-bit asymmetric fixed-point and floating-point types using a quantization parameter calculator, analyzing statistical distributions to determine optimal quantization levels and applying these quantized parameters to enhance both accuracy and dynamic range.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If floating-point type parameters are used in neural network, then accuracy is improved, but power consumption increases and computing speed decreases

Engineering Contradiction:

ImproveaccuracyVSAvoidpower consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent segments parameters into different data types based on their statistical characteristics. Specifically, it divides parameters into symmetric parameters (quantized to fixed-point type) and asymmetric parameters (quantized to asymmetric fixed-point type), allowing each to be processed with the most efficient representation while maintaining overall accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the data type parameter of neural network parameters from uniform floating-point to mixed types (symmetric fixed-point and asymmetric fixed-point). This parameter change optimizes the balance between accuracy and computational efficiency by matching data representation to the actual distribution characteristics of each parameter.

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If floating-point type parameters are used in neural network, then accuracy is improved, but computing speed decreases

Engineering Contradiction:

ImproveaccuracyVSAvoidcomputing speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent segments parameters into different data types based on their statistical characteristics. Specifically, it divides parameters into symmetric parameters (quantized to fixed-point type) and asymmetric parameters (quantized to asymmetric fixed-point type), allowing each to be processed with the most efficient representation while maintaining overall accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the data type parameter of neural network parameters from uniform floating-point to mixed types (symmetric fixed-point and asymmetric fixed-point). This parameter change optimizes the balance between accuracy and computational efficiency by matching data representation to the actual distribution characteristics of each parameter.

Inventive Principle:

Principle #35Parameter changes

3Manufacturing precision

If quantization level is increased to improve accuracy, then manufacturing precision is improved, but device complexity increases

Engineering Contradiction:

Improvequantization precisionVSAvoidquantization complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent applies different quantization strategies to different parameters based on their local characteristics. By analyzing the statistical distribution of each parameter and identifying whether it is symmetric or asymmetric, the system applies the most appropriate quantization method locally, avoiding the need for complex uniform quantization across all parameters.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent changes the data type parameter of neural network parameters from uniform floating-point to mixed types (symmetric fixed-point and asymmetric fixed-point). This parameter change optimizes the balance between accuracy and computational efficiency by matching data representation to the actual distribution characteristics of each parameter.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12073309B2Neural network device and method of quantizing parameters of neural network

Publication Date: 2024.08.27 SAMSUNG ELECTRONICS CO LTD

AI summary

A neural network device includes a quantization parameter calculator configured to quantize parameters of a neural network that is pre-trained, so that the quantized parameters are of mixed data types, analyze a statistical distribution of parameter values of an M-bit floating-point type, the parameter values being associated with at least one layer of the neural network, M being a natural number greater than three, obtain a quantization level of each of the parameters statistically covering a distribution range of the parameter values, based on the analyzed statistical distribution, and quantize input data and weights of the M-bit floating-point type into asymmetric input data of an N-bit fixed-point type and weights of an N-bit floating-point type, using quantization parameters that are obtained based on the obtained quantization level of each of the parameters, N being a natural number greater than one and less than M.