The invention discloses a multi-mechanism mixed
recurrent neural network model compression method. The multi-mechanism mixed
recurrent neural network model compression method comprises A, carrying outcirculant matrix restriction: restricting a part of parameter matrixes in the
recurrent neural network into circulant matrixes, and updating a backward gradient propagation
algorithm to enable the network to carry out
batch training of the circulant matrixes, B, carrying out forward
activation function approximation: replacing a non-linear
activation function with a hardware-friendly linear function during the forward operation process, and keeping the backward gradient updating process unchanged; C, carrying out
hybrid quantization: employing different quantification mechanisms for differentparameters according to the
error tolerance difference between different parameters in the recurrent neural network; and D, employing a secondary training mechanism: dividing
network model training into two phases including initial training and repeated training. Each phase places particular emphasis on a different
model compression method, interaction between different model compression methodsis well avoided, and precision loss brought by the model
compression method is reduced to the maximum extent. According to the invention, a plurality of model compression mechanisms are employed to compress the
recurrent neural network model,
model parameters can be greatly reduced, and the multi-mechanism mixed
recurrent neural network model compression method is suitable for a memory-limited andlow-
delay embedded system needing to use the recurrent neural network, and has good innovativeness and a good application prospect.