Deep neural network hyper-parameter optimization method, electronic device and storage medium

Abstract

The invention discloses a deep neural network hyper-parameter optimization method. The method comprises the steps of using a multi-task conditional neural network model for replacing an existing Gaussian process model to realize a data processing process of hyper-parameter optimization; determining the parameters, sequentially selecting the training sets of a decoder and an encoder according to the set parameters to perform network training, prediction, screening and evaluation; adding the candidate point of each task obtained after evaluation and a target function value into an observation set of the corresponding task; and carrying out the network training, prediction, screening and evaluation again, repeating the above steps until the maximum iterative frequency, and finding out the point, namely the hyper-parameter, when the target function value in the observation set of each task is maximum, thereby solving the problems of complex covariance calculation and the like during the hyper-parameter optimization realized by adopting a Gaussian process in the prior art, and simultaneously realizing the hyper-parameter optimization of multiple tasks. The invention further provides anelectronic device and a storage medium.

Description

technical field [0001] The present invention relates to a hyperparameter optimization method, in particular to a deep neural network hyperparameter optimization method, electronic equipment and a storage medium. Background technique [0002] At present, there are many parameters in the network model of deep learning that are not learned during the training process, but are set directly before the training starts. These parameters are called neural network hyperparameters. However, hyperparameter optimization has always been a difficult problem that limits the performance improvement of network models. At present, neural network models are becoming more and more complex, and there are more and more types of hyperparameters, which often results in the inability to select the appropriate combination of hyperparameters according to the relationship between hyperparameters. At present, the commonly used hyperparameter optimization method is the Bayesian optimization method. This ...

AI Technical Summary

Problems solved by technology

[0003] In order to overcome the deficiencies in the prior art, one of the purposes of the present invention is to provide a deep neural network hyperparameter optimization method, which can solve the problems of high computational cost and long running time in the prior art for hyperparameter optimization

[0004] The second object of the present invention is to p...

Images