Unlock instant, AI-driven research and patent intelligence for your innovation.

A neural network weight compression method based on non-uniform quantization and its application method

A technology of neural network and compression method, applied in the field of neural network weight compression based on non-uniform quantization, can solve the problem of not being able to describe connection weights and connection weights well at the same time, so as to reduce capacity, solve contradictions, and ensure system performance effect

Active Publication Date: 2021-07-09
BEIJING INFORMATION SCI & TECH UNIV
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since the connection weight has the probability density characteristics of Gaussian distribution in the amplitude distribution, uniform quantization cannot describe the connection weight with large amplitude and the connection weight with small amplitude at the same time.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A neural network weight compression method based on non-uniform quantization and its application method
  • A neural network weight compression method based on non-uniform quantization and its application method
  • A neural network weight compression method based on non-uniform quantization and its application method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

[0030] Such as figure 2 As shown, the present invention provides a method for compressing neural network weights based on non-uniform quantization. The method is to group the connection weights obtained after the training of the neural network, normalize the maximum value, and compress and encode. The specific process is as follows:

[0031] 1) Since branch pruning operations are usually performed during the training of connection weights, the weight distribution of the trained neural network presents a double-hump distribution, such as image 3 shown. Therefore, the connection weights are grouped based on the data probability:

[0032] 1.1) The weights are centered on 0 and divided into two groups: group 0 and group 1.

[0033] 1.2) Add the offset value C0 to the weight in group 0, and the offset value is the mean value of group 0, so that the m...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention relates to a neural network weight compression method based on non-uniform quantization and its use method, the steps of which are: grouping connection weights based on data probability; normalizing the maximum value: normalizing the grouped connection weights to the maximum value The normalized connection weights are processed so that the magnitude of the connection weights is in [‑1 1]; the normalized connection weights are compressed and encoded using non-uniform quantized A-law compression. The invention can effectively guarantee the system performance, significantly reduce the capacity required for connection weight storage, and is beneficial to the application of the deep neural network in the embedded system.

Description

technical field [0001] The present invention relates to a deep learning neural network compression method and usage method, in particular to a non-uniform quantization-based neural network weight compression method and usage method applied in the field of computer applications. Background technique [0002] Artificial Neural Network (ANN), as the simplest abstraction and simulation of the human brain, is an intelligent system that imitates the information processing function of the human brain nervous system. It has been a research hotspot in the field of artificial intelligence since the 1980s. . The artificial neural network abstracts the neural network of the human brain from the perspective of mathematical and physical methods and information processing, and establishes a simplified model, aiming to imitate the information processing system of the human brain structure and its functions. [0003] The artificial neural network is composed of multiple neuron connections. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06N3/04H03M7/30
CPCH03M7/3059G06N3/045
Inventor 徐湛张倩文程亚冰张德生
Owner BEIJING INFORMATION SCI & TECH UNIV