Compressed encoding method of sparse neural network

Abstract

The invention relates to a compressed encoding method of a sparse neural network. The method comprises the following steps: firstly, carrying out quantization and preprocessing, and selecting a compression coding method according to a sparsity degree: when the sparsity S is greater than or equal to 2, zero-run and k-order exponential Golomb combined coding is adopted; when the sparsity Epsilon 1<S<Epsilon 2, k-order GX coding is adopted; when the sparsity S is smaller than or equal to Epsilon1, k-order exponential Golomb coding is adopted; wherein the Epsilon1 and the Epsilon2 are set threshold values. The k-order GX coding method comprises the following steps of: if the preprocessing completion data is 0, directly coding into 1 in a binary form, if the preprocessing completion data is greater than 0, representing in the binary form, insufficient k bits and high-order zero padding, and removing the low k bits of the binary bit sequence to obtain a sequence which is converted into a decimal number y; if k bits exist after high-order zero padding, y is 0; calculating a least significant digit LSB in a y + 1 binary form, outputting LSB 0, then outputting a y + 1 bit sequence, and putting the k-bit-removed binary sequence at the output least bit to obtain a coded code word;. According to the method, the distribution characteristics of the sparse neural network weight matrix are utilized, so that the method has higher compression ratio and lower implementation complexity.

Description

technical field [0001] The invention belongs to the technical field of computers, in particular to the field of neural networks, and relates to a compression coding method of a sparse neural network. Background technique [0002] With the advent of the era of artificial intelligence, intelligent tasks such as image recognition, speech recognition, and natural language processing are ubiquitous in life. As one of the most effective algorithms for realizing such intelligent tasks, the neural network has received extensive attention and application in academia and industry. However, a large neural network has a large number of layers and nodes, resulting in a large number of weight parameters, time-consuming network training process, and a trained model takes up a large storage space. The computation-intensive and storage-intensive characteristics of neural networks make it difficult to deploy to mobile and embedded systems with limited resources. Therefore, neural network co...

AI Technical Summary

Problems solved by technology

Most of the existing technologies directly use Huffman coding. This method is more complicated to encode, and at the same time, it is necessary to save a code table before storing or transmitting the coded data for use when the decoder reconstructs the information.

Images