Neural network parameter compression method and related device

Abstract

The invention discloses a neural network parameter compression method, which comprises the following steps: carrying out ascending processing on low-order input data to obtain high-order input data; carrying out tensor decomposition processing on the high-order input data to obtain a high-order output result; and expanding the high-order output result according to the order of the low-order input data to obtain a low-order output result. For the neural network parameter compression method, the low-order input data is upgraded into the high-order input data, then tensor decomposition processing is conducted on the high-order input data, and finally tensor decomposition results are expanded into parameters of the same order of the low-order input data, so that compression of the low-order input data is achieved, the data size during neural network processing is reduced, and the performance utilization rate is increased. The invention further discloses a neural network parameter compression device, computer equipment and a computer readable storage medium which have the above beneficial effects.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a neural network parameter compression method, a neural network parameter compression device, computer equipment, and a computer-readable storage medium. Background technique [0002] With the continuous development of information technology, a variety of parameter processing methods have emerged in the field of deep learning to improve the performance of the deep learning process. [0003] In the prior art, tensor decomposition processing is usually used to perform parameter compression processing on parameters. Generally, tensor decomposition processing includes CP decomposition processing and Tucker decomposition processing. Among them, the Tucker decomposition process is to decompose the tensor into the core tensor and three factor matrices, so as to decompose the parameters of the neural network. When the parameter order of the neural network is higher, the com...

AI Technical Summary

Problems solved by technology

However, when the order of neural network parameters is lower, the compression performance is lower

That is, when Tucker decomposition processing is applied to low-order input data, for example, when Tucker is used to compress first-order parameters, the compression performance is the lowest after compressing the same amount of data, and it cannot be well compressed by Tucker calculations. result

Furthermore, when performing neural network learning on low-order input data, due to the low compression effect, the amount of compression to be processed is high, reducing performance utilization

Images