Automatic searching method and device for precision and decomposition rank of a recurrent neural network

Abstract

The embodiment of the invention discloses an automatic searching method and device for precision and decomposition rank of a recurrent neural network, and the method comprises the steps: initializing a super network, and carrying out the following operation: sampling a sub-network from the super network, and carrying out the performance evaluation of the sampled sub-network; automatically searching a target hyper-parameter combination according to a performance evaluation result, and updating hyper-parameters of the hyper-network according to the target hyper-parameter combination; aiming at the super network after the hyper-parameter adjustment, returning to execute the sub-network sampling operation, and performing the performance evaluation operation on the sampled sub-network; and outputting the current decomposition rank of the super network and the quantized value of each layer of network in response to the condition that the performance evaluation result meets a preset condition. In the embodiment of the invention, according to the performance evaluation result, the decomposition rank of the whole super network and the quantized value corresponding to each layer of network are not searched and updated, so that the purpose of quickly searching the optimal decomposition rank required by model compression and the optimal quantized value of each layer of network is achieved.

Description

technical field [0001] This application relates to the technical field of artificial intelligence, for example, to an automatic search method and device for the accuracy and decomposition rank of a recurrent neural network. Background technique [0002] Deep learning can automatically learn useful features, without relying on feature engineering, and has achieved results that surpass other algorithms in image recognition, video understanding, natural language processing and other tasks. This success is largely due to volume The proposal of Convolution neural network (CNN) and recurrent neural network (Recurrent neural network, RNN). RNN is a type of recursive neural network that takes sequence data as input, recurses in the evolution direction of the sequence, and connects all nodes (recurrent units) in a chain. It is often used to analyze time series information (expression, action, voice, etc.) model middle. However, with the development of artificial intelligence, the c...

AI Technical Summary

Problems solved by technology

If the model tensor is decomposed using the manually set decomposition rank, it will take a lot of manpower and material resources to obtain the optimal decomposition rank

Images