Deep network compression method based on dimensional adaptation Tucker decomposition

Abstract

The invention discloses a deep network compression method based on dimensional adaptation adjusting Tucker decomposition. The method comprises a dimensional adaptation adjusting process and a dimensional adaptation weight tensor decomposition process. Through adaptively adjusting a size of each dimensions of a tensor, a new any order tensor is generated. Through a learnable nuclear tensor and a transfer matrix, tensor decomposition is realized so that a purpose of network optimization compression is reached. Compared to an existing low rank compression method, by using the method of the invention, under the condition that network performance is maintained, a network parameter quantity has a greater compression multiple and a higher compression multiple can be acquired. Simultaneously, a guiding position of a nonzero element does not need to be stored, an index does not need to be recorded and a storage space can be effectively used.

Description

technical field [0001] The invention relates to the technical field of deep learning network optimization compression, in particular to a deep network compression method based on dimension adaptive Tucker decomposition. Background technique [0002] Due to the rapid development of image processor (GPU) computing power in recent years and the increasing amount of data that people can obtain, deep convolutional networks have achieved significant results in the fields of computer vision and natural language processing. According to the development history of the neural network structure (documents [1]~[5]), there are two major trends in the current network structure: the number of layers of the network is getting deeper and deeper, and the convolutional layer is becoming more and more important. While these deeper networks can achieve good results on some problems, their computational cost and storage cost are not a small problem. [0003] Due to the limitations of certain con...

AI Technical Summary

Problems solved by technology

Since the parameters are sparse, the problem with this method is that it needs to record the coordinate guidance of non-zero elements, which will waste a large part of storage space

The problem with this method is that tensor decomposition can only solve fixed-order tensors, and the form of decomposition is fixed. Tensor decomposition is mainly achieved by adding more layers; although this method has achieved good results in network acceleration effect, but due to the limitation of decomposition, the original tensor decomposition method cannot solve the problem of network compression very well

Images